[==========] Running 7 tests from 2 test suites.
[----------] Global test environment set-up.
[----------] 5 tests from LeaderRebalancerTest
[ RUN ] LeaderRebalancerTest.FunctionalTestForDivided
WARNING: Logging before InitGoogleLogging() is written to STDERR
I20260809 15:50:17.290159 9552 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.84.62:42749
I20260809 15:50:17.291368 9552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 15:50:17.292303 9552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:50:17.311022 9560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:50:17.312707 9552 server_base.cc:1034] running on GCE node
W20260809 15:50:17.311076 9559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:50:17.312142 9562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:50:18.501505 9552 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 15:50:18.501750 9552 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 15:50:18.501922 9552 hybrid_clock.cc:648] HybridClock initialized: now 1786290618501900 us; error 0 us; skew 500 ppm
I20260809 15:50:18.502807 9552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:50:18.513792 9552 webserver.cc:460] Webserver started at http://127.9.84.62:41029/ using document root <none> and password file <none>
I20260809 15:50:18.514675 9552 fs_manager.cc:359] Metadata directory not provided
I20260809 15:50:18.514868 9552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:50:18.515251 9552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:50:18.521487 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786290617080470-9552-0/minicluster-data/master-0-root/instance:
uuid: "de4c11a1f6c84b38afc547638849827b"
format_stamp: "Formatted at 2026-08-09 15:50:18 on dist-test-slave-mn4r"
I20260809 15:50:18.529098 9552 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20260809 15:50:18.534159 9568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:50:18.535224 9552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 15:50:18.535504 9552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786290617080470-9552-0/minicluster-data/master-0-root
uuid: "de4c11a1f6c84b38afc547638849827b"
format_stamp: "Formatted at 2026-08-09 15:50:18 on dist-test-slave-mn4r"
I20260809 15:50:18.535799 9552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786290617080470-9552-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786290617080470-9552-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786290617080470-9552-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:50:18.570544 9552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:50:18.571995 9552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 15:50:18.572402 9552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:50:18.644349 9552 rpc_server.cc:307] RPC server started. Bound to: 127.9.84.62:42749
I20260809 15:50:18.644446 9626 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.84.62:42749 every 8 connection(s)
I20260809 15:50:18.651352 9627 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:18.674607 9627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b: Bootstrap starting.
I20260809 15:50:18.680065 9627 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:18.681810 9627 log.cc:824] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b: Log is configured to *not* fsync() on all Append() calls
I20260809 15:50:18.686214 9627 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b: No bootstrap required, opened a new log
I20260809 15:50:18.708508 9627 raft_consensus.cc:348] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de4c11a1f6c84b38afc547638849827b" member_type: VOTER }
I20260809 15:50:18.709025 9627 raft_consensus.cc:374] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:18.709224 9627 raft_consensus.cc:729] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de4c11a1f6c84b38afc547638849827b, State: Initialized, Role: FOLLOWER
I20260809 15:50:18.710034 9627 consensus_queue.cc:260] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de4c11a1f6c84b38afc547638849827b" member_type: VOTER }
I20260809 15:50:18.710517 9627 raft_consensus.cc:388] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:50:18.710748 9627 raft_consensus.cc:482] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:50:18.711009 9627 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:18.715237 9627 raft_consensus.cc:504] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de4c11a1f6c84b38afc547638849827b" member_type: VOTER }
I20260809 15:50:18.715867 9627 leader_election.cc:302] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de4c11a1f6c84b38afc547638849827b; no voters:
I20260809 15:50:18.717590 9627 leader_election.cc:288] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:50:18.717999 9631 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:18.720764 9631 raft_consensus.cc:686] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b [term 1 LEADER]: Becoming Leader. State: Replica: de4c11a1f6c84b38afc547638849827b, State: Running, Role: LEADER
I20260809 15:50:18.721637 9631 consensus_queue.cc:237] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de4c11a1f6c84b38afc547638849827b" member_type: VOTER }
I20260809 15:50:18.722826 9627 sys_catalog.cc:564] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:50:18.731910 9632 sys_catalog.cc:455] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b [sys.catalog]: SysCatalogTable state changed. Reason: New leader de4c11a1f6c84b38afc547638849827b. Latest consensus state: current_term: 1 leader_uuid: "de4c11a1f6c84b38afc547638849827b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de4c11a1f6c84b38afc547638849827b" member_type: VOTER } }
I20260809 15:50:18.732661 9632 sys_catalog.cc:458] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b [sys.catalog]: This master's current role is: LEADER
I20260809 15:50:18.734429 9633 sys_catalog.cc:455] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de4c11a1f6c84b38afc547638849827b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de4c11a1f6c84b38afc547638849827b" member_type: VOTER } }
I20260809 15:50:18.735150 9633 sys_catalog.cc:458] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b [sys.catalog]: This master's current role is: LEADER
I20260809 15:50:18.736208 9639 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:50:18.747475 9639 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:50:18.753378 9552 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260809 15:50:18.765391 9639 catalog_manager.cc:1347] Generated new cluster ID: f3cb4580308e46ecb18b05452051af70
I20260809 15:50:18.765659 9639 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:50:18.784785 9639 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:50:18.786711 9639 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:50:18.805184 9639 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b: Generated new TSK 0
I20260809 15:50:18.806126 9639 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:50:18.820794 9552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:50:18.829823 9650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:50:18.831207 9651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:50:18.834829 9552 server_base.cc:1034] running on GCE node
W20260809 15:50:18.835376 9653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:50:18.836272 9552 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 15:50:18.836499 9552 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 15:50:18.836684 9552 hybrid_clock.cc:648] HybridClock initialized: now 1786290618836670 us; error 0 us; skew 500 ppm
I20260809 15:50:18.837284 9552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:50:18.842309 9552 webserver.cc:460] Webserver started at http://127.9.84.1:38107/ using document root <none> and password file <none>
I20260809 15:50:18.842845 9552 fs_manager.cc:359] Metadata directory not provided
I20260809 15:50:18.843065 9552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:50:18.843354 9552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:50:18.844606 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786290617080470-9552-0/minicluster-data/ts-0-root/instance:
uuid: "042b068763ff48718439228aebb5cfcb"
format_stamp: "Formatted at 2026-08-09 15:50:18 on dist-test-slave-mn4r"
I20260809 15:50:18.850660 9552 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 15:50:18.855381 9658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:50:18.856215 9552 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260809 15:50:18.856487 9552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786290617080470-9552-0/minicluster-data/ts-0-root
uuid: "042b068763ff48718439228aebb5cfcb"
format_stamp: "Formatted at 2026-08-09 15:50:18 on dist-test-slave-mn4r"
I20260809 15:50:18.856775 9552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786290617080470-9552-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786290617080470-9552-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786290617080470-9552-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:50:18.873901 9552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:50:18.875170 9552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:50:18.878019 9552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:50:18.882870 9552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:50:18.883076 9552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:50:18.883383 9552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:50:18.883530 9552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:50:18.992240 9552 rpc_server.cc:307] RPC server started. Bound to: 127.9.84.1:35125
I20260809 15:50:18.992343 9731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.84.1:35125 every 8 connection(s)
I20260809 15:50:19.000265 9552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:50:19.016692 9739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:50:19.017264 9738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:50:19.018709 9741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:50:19.021096 9552 server_base.cc:1034] running on GCE node
I20260809 15:50:19.021934 9552 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 15:50:19.022173 9552 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 15:50:19.022369 9552 hybrid_clock.cc:648] HybridClock initialized: now 1786290619022353 us; error 0 us; skew 500 ppm
I20260809 15:50:19.023169 9552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:50:19.028653 9733 heartbeater.cc:347] Connected to a master server at 127.9.84.62:42749
I20260809 15:50:19.029093 9733 heartbeater.cc:464] Registering TS with master...
I20260809 15:50:19.029322 9552 webserver.cc:460] Webserver started at http://127.9.84.2:44523/ using document root <none> and password file <none>
I20260809 15:50:19.029809 9552 fs_manager.cc:359] Metadata directory not provided
I20260809 15:50:19.029965 9552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:50:19.030238 9552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:50:19.030284 9733 heartbeater.cc:511] Master 127.9.84.62:42749 requested a full tablet report, sending...
I20260809 15:50:19.031360 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786290617080470-9552-0/minicluster-data/ts-1-root/instance:
uuid: "e54dcbceb00649eb944818025b5a969f"
format_stamp: "Formatted at 2026-08-09 15:50:19 on dist-test-slave-mn4r"
I20260809 15:50:19.033776 9587 ts_manager.cc:194] Registered new tserver with Master: 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125)
I20260809 15:50:19.036128 9552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 15:50:19.036825 9587 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60482
I20260809 15:50:19.040638 9746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:50:19.041409 9552 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260809 15:50:19.041697 9552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786290617080470-9552-0/minicluster-data/ts-1-root
uuid: "e54dcbceb00649eb944818025b5a969f"
format_stamp: "Formatted at 2026-08-09 15:50:19 on dist-test-slave-mn4r"
I20260809 15:50:19.041960 9552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786290617080470-9552-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786290617080470-9552-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786290617080470-9552-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:50:19.078161 9552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:50:19.079624 9552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:50:19.082306 9552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:50:19.085727 9552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:50:19.085935 9552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:50:19.086200 9552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:50:19.086340 9552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:50:19.197446 9552 rpc_server.cc:307] RPC server started. Bound to: 127.9.84.2:42899
I20260809 15:50:19.197616 9814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.84.2:42899 every 8 connection(s)
I20260809 15:50:19.208249 9552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:50:19.216295 9818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:50:19.220148 9819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:50:19.223276 9815 heartbeater.cc:347] Connected to a master server at 127.9.84.62:42749
I20260809 15:50:19.223670 9815 heartbeater.cc:464] Registering TS with master...
I20260809 15:50:19.224597 9815 heartbeater.cc:511] Master 127.9.84.62:42749 requested a full tablet report, sending...
I20260809 15:50:19.226848 9587 ts_manager.cc:194] Registered new tserver with Master: e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899)
I20260809 15:50:19.229034 9587 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60490
I20260809 15:50:19.229386 9552 server_base.cc:1034] running on GCE node
W20260809 15:50:19.230605 9822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:50:19.231735 9552 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 15:50:19.232004 9552 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 15:50:19.232203 9552 hybrid_clock.cc:648] HybridClock initialized: now 1786290619232188 us; error 0 us; skew 500 ppm
I20260809 15:50:19.232771 9552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:50:19.235656 9552 webserver.cc:460] Webserver started at http://127.9.84.3:42493/ using document root <none> and password file <none>
I20260809 15:50:19.236287 9552 fs_manager.cc:359] Metadata directory not provided
I20260809 15:50:19.236481 9552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:50:19.236742 9552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:50:19.237802 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786290617080470-9552-0/minicluster-data/ts-2-root/instance:
uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
format_stamp: "Formatted at 2026-08-09 15:50:19 on dist-test-slave-mn4r"
I20260809 15:50:19.242494 9552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.002s
I20260809 15:50:19.245816 9829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:50:19.246665 9552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:50:19.246973 9552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786290617080470-9552-0/minicluster-data/ts-2-root
uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
format_stamp: "Formatted at 2026-08-09 15:50:19 on dist-test-slave-mn4r"
I20260809 15:50:19.247303 9552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786290617080470-9552-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786290617080470-9552-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForDivided.1786290617080470-9552-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:50:19.266007 9552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:50:19.267576 9552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:50:19.269138 9552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:50:19.272782 9552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:50:19.273027 9552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:50:19.273288 9552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:50:19.273496 9552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:50:19.371235 9552 rpc_server.cc:307] RPC server started. Bound to: 127.9.84.3:39553
I20260809 15:50:19.371296 9900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.84.3:39553 every 8 connection(s)
I20260809 15:50:19.389755 9901 heartbeater.cc:347] Connected to a master server at 127.9.84.62:42749
I20260809 15:50:19.390177 9901 heartbeater.cc:464] Registering TS with master...
I20260809 15:50:19.391022 9901 heartbeater.cc:511] Master 127.9.84.62:42749 requested a full tablet report, sending...
I20260809 15:50:19.393054 9587 ts_manager.cc:194] Registered new tserver with Master: 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:19.393276 9552 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.013662301s
I20260809 15:50:19.395171 9587 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:60498
I20260809 15:50:19.396909 9552 test_util.cc:268] Using random seed: 836088601
I20260809 15:50:19.453648 9587 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60508:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q""\004\001\000\216\3438\016\004\001\000\034\307q\034\004\001\000\252\252\252*\004\001\0008\216\3438\004\001\000\306q\034G\004\001\000TUUU\004\001\000\3428\216c\004\001\000p\034\307q"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:50:19.457769 9587 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:50:19.549441 9775 tablet_service.cc:1467] Processing CreateTablet for tablet 3e74d7900d9d485092a9e8b144fca6b0 (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260809 15:50:19.551178 9775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e74d7900d9d485092a9e8b144fca6b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.557727 9774 tablet_service.cc:1467] Processing CreateTablet for tablet 10055574bc224746bde901e394facf6a (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260809 15:50:19.559154 9774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10055574bc224746bde901e394facf6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.559634 9771 tablet_service.cc:1467] Processing CreateTablet for tablet a9d11b0e8c644e089dd34200345495c7 (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260809 15:50:19.560990 9771 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9d11b0e8c644e089dd34200345495c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.567432 9772 tablet_service.cc:1467] Processing CreateTablet for tablet 0fa161c687aa4ee390cca51e913fe50f (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260809 15:50:19.569036 9772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fa161c687aa4ee390cca51e913fe50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.566370 9773 tablet_service.cc:1467] Processing CreateTablet for tablet 726df19d010d4f008eb75311543d0dd6 (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260809 15:50:19.573616 9773 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 726df19d010d4f008eb75311543d0dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.576386 9770 tablet_service.cc:1467] Processing CreateTablet for tablet 90c5ba0a4aea4aebaeecf735050a4528 (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260809 15:50:19.582536 9770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90c5ba0a4aea4aebaeecf735050a4528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.585850 9691 tablet_service.cc:1467] Processing CreateTablet for tablet 3e74d7900d9d485092a9e8b144fca6b0 (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260809 15:50:19.588248 9690 tablet_service.cc:1467] Processing CreateTablet for tablet 10055574bc224746bde901e394facf6a (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260809 15:50:19.591428 9689 tablet_service.cc:1467] Processing CreateTablet for tablet 726df19d010d4f008eb75311543d0dd6 (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260809 15:50:19.584895 9687 tablet_service.cc:1467] Processing CreateTablet for tablet 0fa161c687aa4ee390cca51e913fe50f (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260809 15:50:19.591811 9690 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10055574bc224746bde901e394facf6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.594087 9685 tablet_service.cc:1467] Processing CreateTablet for tablet a9d11b0e8c644e089dd34200345495c7 (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260809 15:50:19.596817 9769 tablet_service.cc:1467] Processing CreateTablet for tablet d9963b7dc24a41838578644fa0572566 (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260809 15:50:19.606521 9684 tablet_service.cc:1467] Processing CreateTablet for tablet 90c5ba0a4aea4aebaeecf735050a4528 (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260809 15:50:19.611377 9682 tablet_service.cc:1467] Processing CreateTablet for tablet d9963b7dc24a41838578644fa0572566 (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260809 15:50:19.613236 9689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 726df19d010d4f008eb75311543d0dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.616739 9687 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fa161c687aa4ee390cca51e913fe50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.621803 9681 tablet_service.cc:1467] Processing CreateTablet for tablet 691afd4416c84d7d8e595a5b7c5e7d4b (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260809 15:50:19.623162 9769 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9963b7dc24a41838578644fa0572566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.626191 9691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e74d7900d9d485092a9e8b144fca6b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.628588 9685 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9d11b0e8c644e089dd34200345495c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.632684 9681 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 691afd4416c84d7d8e595a5b7c5e7d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.638871 9684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90c5ba0a4aea4aebaeecf735050a4528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.676800 9682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9963b7dc24a41838578644fa0572566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.696524 9768 tablet_service.cc:1467] Processing CreateTablet for tablet 691afd4416c84d7d8e595a5b7c5e7d4b (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260809 15:50:19.698053 9768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 691afd4416c84d7d8e595a5b7c5e7d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.707739 9918 tablet_bootstrap.cc:492] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f: Bootstrap starting.
I20260809 15:50:19.715169 9918 tablet_bootstrap.cc:654] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:19.728744 9680 tablet_service.cc:1467] Processing CreateTablet for tablet b7526b699d7d4b9680c8f2e98b9c72e2 (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260809 15:50:19.730830 9680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7526b699d7d4b9680c8f2e98b9c72e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.737747 9772 tablet_service.cc:1467] Processing CreateTablet for tablet b7526b699d7d4b9680c8f2e98b9c72e2 (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260809 15:50:19.743338 9772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7526b699d7d4b9680c8f2e98b9c72e2. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 15:50:19.745625 9641 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 15:50:19.788363 9858 tablet_service.cc:1467] Processing CreateTablet for tablet 726df19d010d4f008eb75311543d0dd6 (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 477218588 <= VALUES < 715827882
I20260809 15:50:19.790212 9857 tablet_service.cc:1467] Processing CreateTablet for tablet 0fa161c687aa4ee390cca51e913fe50f (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 715827882 <= VALUES < 954437176
I20260809 15:50:19.793398 9858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 726df19d010d4f008eb75311543d0dd6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.783787 9861 tablet_service.cc:1467] Processing CreateTablet for tablet 3e74d7900d9d485092a9e8b144fca6b0 (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION VALUES < 238609294
I20260809 15:50:19.783870 9860 tablet_service.cc:1467] Processing CreateTablet for tablet 10055574bc224746bde901e394facf6a (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 238609294 <= VALUES < 477218588
I20260809 15:50:19.804458 9857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0fa161c687aa4ee390cca51e913fe50f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.808538 9856 tablet_service.cc:1467] Processing CreateTablet for tablet a9d11b0e8c644e089dd34200345495c7 (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 954437176 <= VALUES < 1193046470
I20260809 15:50:19.809619 9855 tablet_service.cc:1467] Processing CreateTablet for tablet 90c5ba0a4aea4aebaeecf735050a4528 (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 1193046470 <= VALUES < 1431655764
I20260809 15:50:19.805830 9853 tablet_service.cc:1467] Processing CreateTablet for tablet d9963b7dc24a41838578644fa0572566 (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 1431655764 <= VALUES < 1670265058
I20260809 15:50:19.821182 9918 tablet_bootstrap.cc:492] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f: No bootstrap required, opened a new log
I20260809 15:50:19.819041 9852 tablet_service.cc:1467] Processing CreateTablet for tablet 691afd4416c84d7d8e595a5b7c5e7d4b (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 1670265058 <= VALUES < 1908874352
I20260809 15:50:19.823668 9851 tablet_service.cc:1467] Processing CreateTablet for tablet b7526b699d7d4b9680c8f2e98b9c72e2 (DEFAULT_TABLE table=test-workload [id=3aed9bcf8a464b938e61bebf863500f0]), partition=RANGE (key) PARTITION 1908874352 <= VALUES
I20260809 15:50:19.823011 9860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10055574bc224746bde901e394facf6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.825991 9918 ts_tablet_manager.cc:1397] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f: Time spent bootstrapping tablet: real 0.119s user 0.014s sys 0.023s
I20260809 15:50:19.829195 9855 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 90c5ba0a4aea4aebaeecf735050a4528. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.831822 9851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7526b699d7d4b9680c8f2e98b9c72e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.833550 9853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9963b7dc24a41838578644fa0572566. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.837387 9856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a9d11b0e8c644e089dd34200345495c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.841578 9861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e74d7900d9d485092a9e8b144fca6b0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.842839 9852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 691afd4416c84d7d8e595a5b7c5e7d4b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:19.855976 9920 tablet_bootstrap.cc:492] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb: Bootstrap starting.
I20260809 15:50:19.912115 9918 raft_consensus.cc:348] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } }
I20260809 15:50:19.924363 9918 raft_consensus.cc:374] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:19.925149 9918 raft_consensus.cc:729] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Initialized, Role: FOLLOWER
I20260809 15:50:19.932651 9918 consensus_queue.cc:260] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } }
I20260809 15:50:19.943382 9924 raft_consensus.cc:482] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:19.943435 9815 heartbeater.cc:503] Master 127.9.84.62:42749 was elected leader, sending a full tablet report...
I20260809 15:50:19.943848 9924 raft_consensus.cc:504] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } }
I20260809 15:50:19.948697 9921 tablet_bootstrap.cc:492] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7: Bootstrap starting.
I20260809 15:50:19.951951 9918 ts_tablet_manager.cc:1428] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f: Time spent starting tablet: real 0.123s user 0.011s sys 0.007s
I20260809 15:50:19.952510 9924 leader_election.cc:288] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553), 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125)
I20260809 15:50:19.957168 9918 tablet_bootstrap.cc:492] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f: Bootstrap starting.
W20260809 15:50:19.968029 9816 tablet.cc:2369] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:50:19.986361 9920 tablet_bootstrap.cc:654] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.004673 9921 tablet_bootstrap.cc:654] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.010347 9920 tablet_bootstrap.cc:492] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb: No bootstrap required, opened a new log
I20260809 15:50:20.011300 9920 ts_tablet_manager.cc:1397] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb: Time spent bootstrapping tablet: real 0.197s user 0.006s sys 0.015s
I20260809 15:50:20.016490 9920 raft_consensus.cc:348] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.019222 9920 raft_consensus.cc:374] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.019557 9920 raft_consensus.cc:729] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.020279 9920 consensus_queue.cc:260] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.021142 9918 tablet_bootstrap.cc:654] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.035995 9704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e74d7900d9d485092a9e8b144fca6b0" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "042b068763ff48718439228aebb5cfcb" is_pre_election: true
I20260809 15:50:20.037393 9918 tablet_bootstrap.cc:492] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f: No bootstrap required, opened a new log
I20260809 15:50:20.037879 9918 ts_tablet_manager.cc:1397] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f: Time spent bootstrapping tablet: real 0.081s user 0.011s sys 0.007s
W20260809 15:50:20.037964 9747 leader_election.cc:341] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:20.039940 9921 tablet_bootstrap.cc:492] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7: No bootstrap required, opened a new log
I20260809 15:50:20.040374 9921 ts_tablet_manager.cc:1397] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7: Time spent bootstrapping tablet: real 0.099s user 0.009s sys 0.009s
I20260809 15:50:20.040958 9733 heartbeater.cc:503] Master 127.9.84.62:42749 was elected leader, sending a full tablet report...
I20260809 15:50:20.043963 9921 raft_consensus.cc:348] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.046072 9874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e74d7900d9d485092a9e8b144fca6b0" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" is_pre_election: true
I20260809 15:50:20.046577 9918 raft_consensus.cc:348] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
W20260809 15:50:20.047772 9749 leader_election.cc:341] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:20.048213 9749 leader_election.cc:302] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e54dcbceb00649eb944818025b5a969f; no voters: 042b068763ff48718439228aebb5cfcb, 4a0709a93f76476a8acbfa4e9eb955d7
I20260809 15:50:20.044817 9921 raft_consensus.cc:374] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.049427 9923 raft_consensus.cc:2726] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:50:20.047288 9918 raft_consensus.cc:374] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.050328 9918 raft_consensus.cc:729] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.051026 9918 consensus_queue.cc:260] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.053424 9918 ts_tablet_manager.cc:1428] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f: Time spent starting tablet: real 0.015s user 0.004s sys 0.001s
I20260809 15:50:20.054286 9918 tablet_bootstrap.cc:492] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f: Bootstrap starting.
I20260809 15:50:20.057317 9921 raft_consensus.cc:729] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a0709a93f76476a8acbfa4e9eb955d7, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.058432 9921 consensus_queue.cc:260] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.061043 9918 tablet_bootstrap.cc:654] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.071707 9901 heartbeater.cc:503] Master 127.9.84.62:42749 was elected leader, sending a full tablet report...
I20260809 15:50:20.080773 9929 raft_consensus.cc:482] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:20.081295 9920 ts_tablet_manager.cc:1428] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb: Time spent starting tablet: real 0.069s user 0.034s sys 0.024s
I20260809 15:50:20.082329 9920 tablet_bootstrap.cc:492] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb: Bootstrap starting.
I20260809 15:50:20.084733 9921 ts_tablet_manager.cc:1428] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7: Time spent starting tablet: real 0.044s user 0.005s sys 0.003s
I20260809 15:50:20.086733 9921 tablet_bootstrap.cc:492] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7: Bootstrap starting.
I20260809 15:50:20.088487 9920 tablet_bootstrap.cc:654] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.088835 9931 raft_consensus.cc:482] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:20.089306 9931 raft_consensus.cc:504] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.081403 9929 raft_consensus.cc:504] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.092625 9929 leader_election.cc:288] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.106001 9931 leader_election.cc:288] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899)
I20260809 15:50:20.121904 9920 tablet_bootstrap.cc:492] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb: No bootstrap required, opened a new log
I20260809 15:50:20.122430 9920 ts_tablet_manager.cc:1397] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb: Time spent bootstrapping tablet: real 0.040s user 0.013s sys 0.005s
I20260809 15:50:20.125223 9920 raft_consensus.cc:348] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.126394 9923 raft_consensus.cc:482] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:20.126847 9923 raft_consensus.cc:504] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.129074 9923 leader_election.cc:288] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.129788 9704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fa161c687aa4ee390cca51e913fe50f" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "042b068763ff48718439228aebb5cfcb" is_pre_election: true
I20260809 15:50:20.130188 9921 tablet_bootstrap.cc:654] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7: Neither blocks nor log segments found. Creating new log.
W20260809 15:50:20.131187 9747 leader_election.cc:341] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:20.131902 9918 tablet_bootstrap.cc:492] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f: No bootstrap required, opened a new log
I20260809 15:50:20.132371 9918 ts_tablet_manager.cc:1397] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f: Time spent bootstrapping tablet: real 0.078s user 0.004s sys 0.032s
I20260809 15:50:20.135825 9873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691afd4416c84d7d8e595a5b7c5e7d4b" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" is_pre_election: true
I20260809 15:50:20.136214 9918 raft_consensus.cc:348] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.137197 9918 raft_consensus.cc:374] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260809 15:50:20.137341 9661 leader_election.cc:341] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:20.137591 9918 raft_consensus.cc:729] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.138553 9874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fa161c687aa4ee390cca51e913fe50f" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" is_pre_election: true
W20260809 15:50:20.140148 9749 leader_election.cc:341] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:20.140530 9749 leader_election.cc:302] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e54dcbceb00649eb944818025b5a969f; no voters: 042b068763ff48718439228aebb5cfcb, 4a0709a93f76476a8acbfa4e9eb955d7
I20260809 15:50:20.141234 9923 raft_consensus.cc:2726] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:50:20.141958 9918 consensus_queue.cc:260] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.144080 9704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9d11b0e8c644e089dd34200345495c7" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "042b068763ff48718439228aebb5cfcb" is_pre_election: true
I20260809 15:50:20.144397 9918 ts_tablet_manager.cc:1428] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f: Time spent starting tablet: real 0.012s user 0.010s sys 0.000s
I20260809 15:50:20.145252 9918 tablet_bootstrap.cc:492] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f: Bootstrap starting.
I20260809 15:50:20.125983 9920 raft_consensus.cc:374] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.148149 9920 raft_consensus.cc:729] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.149780 9921 tablet_bootstrap.cc:492] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7: No bootstrap required, opened a new log
I20260809 15:50:20.150714 9921 ts_tablet_manager.cc:1397] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7: Time spent bootstrapping tablet: real 0.064s user 0.016s sys 0.004s
I20260809 15:50:20.152258 9920 consensus_queue.cc:260] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.154424 9704 raft_consensus.cc:2454] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a0709a93f76476a8acbfa4e9eb955d7 in term 0.
I20260809 15:50:20.157373 9921 raft_consensus.cc:348] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.160565 9920 ts_tablet_manager.cc:1428] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb: Time spent starting tablet: real 0.038s user 0.007s sys 0.000s
I20260809 15:50:20.164345 9920 tablet_bootstrap.cc:492] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb: Bootstrap starting.
I20260809 15:50:20.164703 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9d11b0e8c644e089dd34200345495c7" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e54dcbceb00649eb944818025b5a969f" is_pre_election: true
I20260809 15:50:20.169337 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691afd4416c84d7d8e595a5b7c5e7d4b" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e54dcbceb00649eb944818025b5a969f" is_pre_election: true
I20260809 15:50:20.170590 9920 tablet_bootstrap.cc:654] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb: Neither blocks nor log segments found. Creating new log.
W20260809 15:50:20.170639 9661 leader_election.cc:341] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:20.171164 9661 leader_election.cc:302] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 042b068763ff48718439228aebb5cfcb; no voters: 4a0709a93f76476a8acbfa4e9eb955d7, e54dcbceb00649eb944818025b5a969f
I20260809 15:50:20.172425 9933 raft_consensus.cc:2726] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:50:20.160332 9918 tablet_bootstrap.cc:654] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.160630 9921 raft_consensus.cc:374] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.158535 9830 leader_election.cc:302] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, 4a0709a93f76476a8acbfa4e9eb955d7; no voters:
I20260809 15:50:20.175483 9931 raft_consensus.cc:2781] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260809 15:50:20.175760 9832 leader_election.cc:341] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:20.175959 9931 raft_consensus.cc:482] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:50:20.176353 9931 raft_consensus.cc:3037] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.174962 9921 raft_consensus.cc:729] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a0709a93f76476a8acbfa4e9eb955d7, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.183171 9920 tablet_bootstrap.cc:492] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb: No bootstrap required, opened a new log
I20260809 15:50:20.182685 9921 consensus_queue.cc:260] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.183629 9920 ts_tablet_manager.cc:1397] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.007s
I20260809 15:50:20.183630 9931 raft_consensus.cc:504] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.186399 9931 leader_election.cc:288] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 1 election: Requested vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899)
I20260809 15:50:20.187420 9704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9d11b0e8c644e089dd34200345495c7" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "042b068763ff48718439228aebb5cfcb"
I20260809 15:50:20.187635 9918 tablet_bootstrap.cc:492] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f: No bootstrap required, opened a new log
I20260809 15:50:20.187757 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9d11b0e8c644e089dd34200345495c7" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e54dcbceb00649eb944818025b5a969f"
I20260809 15:50:20.188019 9704 raft_consensus.cc:3037] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.188138 9918 ts_tablet_manager.cc:1397] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f: Time spent bootstrapping tablet: real 0.043s user 0.009s sys 0.007s
I20260809 15:50:20.190953 9918 raft_consensus.cc:348] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.191676 9918 raft_consensus.cc:374] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.191944 9918 raft_consensus.cc:729] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Initialized, Role: FOLLOWER
W20260809 15:50:20.192831 9832 leader_election.cc:341] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:20.192544 9918 consensus_queue.cc:260] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.194651 9704 raft_consensus.cc:2454] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a0709a93f76476a8acbfa4e9eb955d7 in term 1.
I20260809 15:50:20.194931 9918 ts_tablet_manager.cc:1428] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260809 15:50:20.195766 9918 tablet_bootstrap.cc:492] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f: Bootstrap starting.
I20260809 15:50:20.195995 9830 leader_election.cc:302] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, 4a0709a93f76476a8acbfa4e9eb955d7; no voters: e54dcbceb00649eb944818025b5a969f
I20260809 15:50:20.196770 9931 raft_consensus.cc:2781] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:20.197223 9931 raft_consensus.cc:686] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 LEADER]: Becoming Leader. State: Replica: 4a0709a93f76476a8acbfa4e9eb955d7, State: Running, Role: LEADER
I20260809 15:50:20.196967 9920 raft_consensus.cc:348] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.197674 9920 raft_consensus.cc:374] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.197957 9920 raft_consensus.cc:729] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.198017 9931 consensus_queue.cc:237] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.198731 9921 ts_tablet_manager.cc:1428] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7: Time spent starting tablet: real 0.048s user 0.009s sys 0.001s
I20260809 15:50:20.198750 9920 consensus_queue.cc:260] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.199709 9921 tablet_bootstrap.cc:492] T d9963b7dc24a41838578644fa0572566 P 4a0709a93f76476a8acbfa4e9eb955d7: Bootstrap starting.
I20260809 15:50:20.201367 9920 ts_tablet_manager.cc:1428] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb: Time spent starting tablet: real 0.017s user 0.004s sys 0.004s
I20260809 15:50:20.202317 9920 tablet_bootstrap.cc:492] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb: Bootstrap starting.
I20260809 15:50:20.208585 9920 tablet_bootstrap.cc:654] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.211278 9918 tablet_bootstrap.cc:654] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.217392 9921 tablet_bootstrap.cc:654] T d9963b7dc24a41838578644fa0572566 P 4a0709a93f76476a8acbfa4e9eb955d7: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.227304 9918 tablet_bootstrap.cc:492] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f: No bootstrap required, opened a new log
I20260809 15:50:20.227746 9918 ts_tablet_manager.cc:1397] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f: Time spent bootstrapping tablet: real 0.032s user 0.007s sys 0.006s
I20260809 15:50:20.229238 9920 tablet_bootstrap.cc:492] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb: No bootstrap required, opened a new log
I20260809 15:50:20.229702 9920 ts_tablet_manager.cc:1397] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb: Time spent bootstrapping tablet: real 0.028s user 0.018s sys 0.005s
I20260809 15:50:20.233283 9918 raft_consensus.cc:348] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.234040 9918 raft_consensus.cc:374] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.234447 9918 raft_consensus.cc:729] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.234164 9920 raft_consensus.cc:348] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.234943 9920 raft_consensus.cc:374] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.235312 9920 raft_consensus.cc:729] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.233899 9587 catalog_manager.cc:5495] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3). New cstate: current_term: 1 leader_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: HEALTHY } } }
I20260809 15:50:20.235666 9918 consensus_queue.cc:260] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.238855 9918 ts_tablet_manager.cc:1428] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f: Time spent starting tablet: real 0.011s user 0.004s sys 0.003s
I20260809 15:50:20.235951 9920 consensus_queue.cc:260] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.242321 9920 ts_tablet_manager.cc:1428] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb: Time spent starting tablet: real 0.012s user 0.008s sys 0.000s
I20260809 15:50:20.243057 9918 tablet_bootstrap.cc:492] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f: Bootstrap starting.
I20260809 15:50:20.243196 9920 tablet_bootstrap.cc:492] T b7526b699d7d4b9680c8f2e98b9c72e2 P 042b068763ff48718439228aebb5cfcb: Bootstrap starting.
I20260809 15:50:20.245978 9921 tablet_bootstrap.cc:492] T d9963b7dc24a41838578644fa0572566 P 4a0709a93f76476a8acbfa4e9eb955d7: No bootstrap required, opened a new log
I20260809 15:50:20.246431 9921 ts_tablet_manager.cc:1397] T d9963b7dc24a41838578644fa0572566 P 4a0709a93f76476a8acbfa4e9eb955d7: Time spent bootstrapping tablet: real 0.047s user 0.007s sys 0.014s
I20260809 15:50:20.249223 9920 tablet_bootstrap.cc:654] T b7526b699d7d4b9680c8f2e98b9c72e2 P 042b068763ff48718439228aebb5cfcb: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.249166 9921 raft_consensus.cc:348] T d9963b7dc24a41838578644fa0572566 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.251762 9921 raft_consensus.cc:374] T d9963b7dc24a41838578644fa0572566 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.252050 9921 raft_consensus.cc:729] T d9963b7dc24a41838578644fa0572566 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a0709a93f76476a8acbfa4e9eb955d7, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.255278 9921 consensus_queue.cc:260] T d9963b7dc24a41838578644fa0572566 P 4a0709a93f76476a8acbfa4e9eb955d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.262295 9918 tablet_bootstrap.cc:654] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.264906 9921 ts_tablet_manager.cc:1428] T d9963b7dc24a41838578644fa0572566 P 4a0709a93f76476a8acbfa4e9eb955d7: Time spent starting tablet: real 0.018s user 0.006s sys 0.002s
I20260809 15:50:20.265931 9921 tablet_bootstrap.cc:492] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7: Bootstrap starting.
I20260809 15:50:20.268625 9920 tablet_bootstrap.cc:492] T b7526b699d7d4b9680c8f2e98b9c72e2 P 042b068763ff48718439228aebb5cfcb: No bootstrap required, opened a new log
I20260809 15:50:20.269035 9920 ts_tablet_manager.cc:1397] T b7526b699d7d4b9680c8f2e98b9c72e2 P 042b068763ff48718439228aebb5cfcb: Time spent bootstrapping tablet: real 0.026s user 0.009s sys 0.014s
I20260809 15:50:20.271776 9920 raft_consensus.cc:348] T b7526b699d7d4b9680c8f2e98b9c72e2 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.272415 9920 raft_consensus.cc:374] T b7526b699d7d4b9680c8f2e98b9c72e2 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.272686 9920 raft_consensus.cc:729] T b7526b699d7d4b9680c8f2e98b9c72e2 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.273274 9920 consensus_queue.cc:260] T b7526b699d7d4b9680c8f2e98b9c72e2 P 042b068763ff48718439228aebb5cfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.273805 9921 tablet_bootstrap.cc:654] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.275317 9920 ts_tablet_manager.cc:1428] T b7526b699d7d4b9680c8f2e98b9c72e2 P 042b068763ff48718439228aebb5cfcb: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260809 15:50:20.276154 9920 tablet_bootstrap.cc:492] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb: Bootstrap starting.
I20260809 15:50:20.281502 9920 tablet_bootstrap.cc:654] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.284183 9918 tablet_bootstrap.cc:492] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f: No bootstrap required, opened a new log
I20260809 15:50:20.284679 9918 ts_tablet_manager.cc:1397] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f: Time spent bootstrapping tablet: real 0.044s user 0.002s sys 0.008s
I20260809 15:50:20.288060 9918 raft_consensus.cc:348] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.288995 9918 raft_consensus.cc:374] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.289340 9918 raft_consensus.cc:729] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.290300 9918 consensus_queue.cc:260] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.293226 9918 ts_tablet_manager.cc:1428] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260809 15:50:20.294378 9918 tablet_bootstrap.cc:492] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f: Bootstrap starting.
I20260809 15:50:20.297487 9921 tablet_bootstrap.cc:492] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7: No bootstrap required, opened a new log
I20260809 15:50:20.299147 9921 ts_tablet_manager.cc:1397] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7: Time spent bootstrapping tablet: real 0.033s user 0.023s sys 0.008s
I20260809 15:50:20.299500 9920 tablet_bootstrap.cc:492] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb: No bootstrap required, opened a new log
I20260809 15:50:20.299952 9920 ts_tablet_manager.cc:1397] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb: Time spent bootstrapping tablet: real 0.024s user 0.009s sys 0.012s
I20260809 15:50:20.302558 9920 raft_consensus.cc:348] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.303256 9920 raft_consensus.cc:374] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.303546 9920 raft_consensus.cc:729] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.304056 9918 tablet_bootstrap.cc:654] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.303666 9921 raft_consensus.cc:348] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.304540 9921 raft_consensus.cc:374] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.304232 9920 consensus_queue.cc:260] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.304847 9921 raft_consensus.cc:729] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a0709a93f76476a8acbfa4e9eb955d7, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.305814 9921 consensus_queue.cc:260] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.306797 9920 ts_tablet_manager.cc:1428] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 15:50:20.307730 9920 tablet_bootstrap.cc:492] T 90c5ba0a4aea4aebaeecf735050a4528 P 042b068763ff48718439228aebb5cfcb: Bootstrap starting.
I20260809 15:50:20.308251 9921 ts_tablet_manager.cc:1428] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260809 15:50:20.309286 9921 tablet_bootstrap.cc:492] T 10055574bc224746bde901e394facf6a P 4a0709a93f76476a8acbfa4e9eb955d7: Bootstrap starting.
I20260809 15:50:20.323050 9921 tablet_bootstrap.cc:654] T 10055574bc224746bde901e394facf6a P 4a0709a93f76476a8acbfa4e9eb955d7: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.326767 9920 tablet_bootstrap.cc:654] T 90c5ba0a4aea4aebaeecf735050a4528 P 042b068763ff48718439228aebb5cfcb: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.332158 9918 tablet_bootstrap.cc:492] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f: No bootstrap required, opened a new log
I20260809 15:50:20.332789 9918 ts_tablet_manager.cc:1397] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f: Time spent bootstrapping tablet: real 0.039s user 0.007s sys 0.011s
I20260809 15:50:20.335942 9921 tablet_bootstrap.cc:492] T 10055574bc224746bde901e394facf6a P 4a0709a93f76476a8acbfa4e9eb955d7: No bootstrap required, opened a new log
I20260809 15:50:20.336409 9921 ts_tablet_manager.cc:1397] T 10055574bc224746bde901e394facf6a P 4a0709a93f76476a8acbfa4e9eb955d7: Time spent bootstrapping tablet: real 0.027s user 0.012s sys 0.006s
I20260809 15:50:20.337164 9918 raft_consensus.cc:348] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.338032 9918 raft_consensus.cc:374] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.338222 9933 raft_consensus.cc:482] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:20.338392 9918 raft_consensus.cc:729] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.338725 9933 raft_consensus.cc:504] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.339221 9918 consensus_queue.cc:260] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.339711 9921 raft_consensus.cc:348] T 10055574bc224746bde901e394facf6a P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.340296 9920 tablet_bootstrap.cc:492] T 90c5ba0a4aea4aebaeecf735050a4528 P 042b068763ff48718439228aebb5cfcb: No bootstrap required, opened a new log
I20260809 15:50:20.341727 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9963b7dc24a41838578644fa0572566" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e54dcbceb00649eb944818025b5a969f" is_pre_election: true
I20260809 15:50:20.342846 9920 ts_tablet_manager.cc:1397] T 90c5ba0a4aea4aebaeecf735050a4528 P 042b068763ff48718439228aebb5cfcb: Time spent bootstrapping tablet: real 0.035s user 0.012s sys 0.008s
I20260809 15:50:20.343137 9788 raft_consensus.cc:2454] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 042b068763ff48718439228aebb5cfcb in term 0.
I20260809 15:50:20.344462 9661 leader_election.cc:302] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, e54dcbceb00649eb944818025b5a969f; no voters:
I20260809 15:50:20.345603 9920 raft_consensus.cc:348] T 90c5ba0a4aea4aebaeecf735050a4528 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.346252 9929 raft_consensus.cc:2781] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:50:20.341976 9874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9963b7dc24a41838578644fa0572566" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" is_pre_election: true
I20260809 15:50:20.346673 9929 raft_consensus.cc:482] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:50:20.347009 9929 raft_consensus.cc:3037] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.346947 9874 raft_consensus.cc:2454] T d9963b7dc24a41838578644fa0572566 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 042b068763ff48718439228aebb5cfcb in term 0.
I20260809 15:50:20.340540 9921 raft_consensus.cc:374] T 10055574bc224746bde901e394facf6a P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.340713 9933 leader_election.cc:288] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.348206 9921 raft_consensus.cc:729] T 10055574bc224746bde901e394facf6a P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a0709a93f76476a8acbfa4e9eb955d7, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.346269 9920 raft_consensus.cc:374] T 90c5ba0a4aea4aebaeecf735050a4528 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.349189 9920 raft_consensus.cc:729] T 90c5ba0a4aea4aebaeecf735050a4528 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.349843 9920 consensus_queue.cc:260] T 90c5ba0a4aea4aebaeecf735050a4528 P 042b068763ff48718439228aebb5cfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.350244 9921 consensus_queue.cc:260] T 10055574bc224746bde901e394facf6a P 4a0709a93f76476a8acbfa4e9eb955d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.352236 9920 ts_tablet_manager.cc:1428] T 90c5ba0a4aea4aebaeecf735050a4528 P 042b068763ff48718439228aebb5cfcb: Time spent starting tablet: real 0.009s user 0.005s sys 0.000s
I20260809 15:50:20.352626 9921 ts_tablet_manager.cc:1428] T 10055574bc224746bde901e394facf6a P 4a0709a93f76476a8acbfa4e9eb955d7: Time spent starting tablet: real 0.016s user 0.006s sys 0.001s
I20260809 15:50:20.353102 9920 tablet_bootstrap.cc:492] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb: Bootstrap starting.
I20260809 15:50:20.353647 9921 tablet_bootstrap.cc:492] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7: Bootstrap starting.
I20260809 15:50:20.359447 9920 tablet_bootstrap.cc:654] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.359838 9921 tablet_bootstrap.cc:654] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.369062 9921 tablet_bootstrap.cc:492] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7: No bootstrap required, opened a new log
I20260809 15:50:20.369560 9921 ts_tablet_manager.cc:1397] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7: Time spent bootstrapping tablet: real 0.016s user 0.011s sys 0.003s
I20260809 15:50:20.374819 9921 raft_consensus.cc:348] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.376997 9921 raft_consensus.cc:374] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.377372 9921 raft_consensus.cc:729] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a0709a93f76476a8acbfa4e9eb955d7, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.341892 9918 ts_tablet_manager.cc:1428] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f: Time spent starting tablet: real 0.009s user 0.003s sys 0.007s
I20260809 15:50:20.378202 9921 consensus_queue.cc:260] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.378886 9920 tablet_bootstrap.cc:492] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb: No bootstrap required, opened a new log
I20260809 15:50:20.376704 9929 raft_consensus.cc:504] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.379467 9920 ts_tablet_manager.cc:1397] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb: Time spent bootstrapping tablet: real 0.027s user 0.012s sys 0.012s
I20260809 15:50:20.381597 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9963b7dc24a41838578644fa0572566" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e54dcbceb00649eb944818025b5a969f"
I20260809 15:50:20.382203 9788 raft_consensus.cc:3037] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.382201 9920 raft_consensus.cc:348] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.382959 9920 raft_consensus.cc:374] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.383219 9920 raft_consensus.cc:729] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.383812 9920 consensus_queue.cc:260] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.385797 9920 ts_tablet_manager.cc:1428] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260809 15:50:20.387226 9920 tablet_bootstrap.cc:492] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb: Bootstrap starting.
I20260809 15:50:20.391101 9788 raft_consensus.cc:2454] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 042b068763ff48718439228aebb5cfcb in term 1.
I20260809 15:50:20.392232 9661 leader_election.cc:302] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, e54dcbceb00649eb944818025b5a969f; no voters:
I20260809 15:50:20.392489 9920 tablet_bootstrap.cc:654] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.392940 9933 raft_consensus.cc:2781] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:20.394572 9945 raft_consensus.cc:482] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:20.394574 9874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9963b7dc24a41838578644fa0572566" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
I20260809 15:50:20.395274 9874 raft_consensus.cc:3037] T d9963b7dc24a41838578644fa0572566 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.395140 9945 raft_consensus.cc:504] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.397289 9945 leader_election.cc:288] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899)
I20260809 15:50:20.398234 9929 leader_election.cc:288] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 election: Requested vote from peers e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.398813 9704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691afd4416c84d7d8e595a5b7c5e7d4b" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "042b068763ff48718439228aebb5cfcb" is_pre_election: true
I20260809 15:50:20.401131 9704 raft_consensus.cc:2454] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a0709a93f76476a8acbfa4e9eb955d7 in term 0.
I20260809 15:50:20.399047 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691afd4416c84d7d8e595a5b7c5e7d4b" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e54dcbceb00649eb944818025b5a969f" is_pre_election: true
I20260809 15:50:20.397454 9918 tablet_bootstrap.cc:492] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f: Bootstrap starting.
I20260809 15:50:20.403676 9874 raft_consensus.cc:2454] T d9963b7dc24a41838578644fa0572566 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 042b068763ff48718439228aebb5cfcb in term 1.
I20260809 15:50:20.400444 9921 ts_tablet_manager.cc:1428] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7: Time spent starting tablet: real 0.031s user 0.006s sys 0.000s
I20260809 15:50:20.402248 9830 leader_election.cc:302] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, 4a0709a93f76476a8acbfa4e9eb955d7; no voters:
I20260809 15:50:20.407804 9945 raft_consensus.cc:2781] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:50:20.408236 9945 raft_consensus.cc:482] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 15:50:20.408319 9832 leader_election.cc:341] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:20.408586 9945 raft_consensus.cc:3037] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.408710 9918 tablet_bootstrap.cc:654] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.409857 9921 tablet_bootstrap.cc:492] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7: Bootstrap starting.
I20260809 15:50:20.415997 9921 tablet_bootstrap.cc:654] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.417702 9945 raft_consensus.cc:504] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.419219 9920 tablet_bootstrap.cc:492] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb: No bootstrap required, opened a new log
I20260809 15:50:20.422448 9920 ts_tablet_manager.cc:1397] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb: Time spent bootstrapping tablet: real 0.035s user 0.008s sys 0.003s
I20260809 15:50:20.424860 9920 raft_consensus.cc:348] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } }
I20260809 15:50:20.425483 9920 raft_consensus.cc:374] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.425761 9920 raft_consensus.cc:729] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.426429 9920 consensus_queue.cc:260] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } }
I20260809 15:50:20.428308 9920 ts_tablet_manager.cc:1428] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260809 15:50:20.419483 9933 raft_consensus.cc:686] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 1 LEADER]: Becoming Leader. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Running, Role: LEADER
I20260809 15:50:20.428622 9918 tablet_bootstrap.cc:492] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f: No bootstrap required, opened a new log
I20260809 15:50:20.429246 9918 ts_tablet_manager.cc:1397] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f: Time spent bootstrapping tablet: real 0.032s user 0.007s sys 0.009s
I20260809 15:50:20.429322 9933 consensus_queue.cc:237] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.432996 9945 leader_election.cc:288] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 1 election: Requested vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899)
I20260809 15:50:20.434028 9704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691afd4416c84d7d8e595a5b7c5e7d4b" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "042b068763ff48718439228aebb5cfcb"
I20260809 15:50:20.434028 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691afd4416c84d7d8e595a5b7c5e7d4b" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e54dcbceb00649eb944818025b5a969f"
I20260809 15:50:20.434871 9704 raft_consensus.cc:3037] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:50:20.435698 9832 leader_election.cc:341] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:20.440745 9962 raft_consensus.cc:482] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:20.441180 9962 raft_consensus.cc:504] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } }
I20260809 15:50:20.444660 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e74d7900d9d485092a9e8b144fca6b0" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e54dcbceb00649eb944818025b5a969f" is_pre_election: true
I20260809 15:50:20.445327 9788 raft_consensus.cc:2454] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 042b068763ff48718439228aebb5cfcb in term 0.
I20260809 15:50:20.445466 9874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e74d7900d9d485092a9e8b144fca6b0" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" is_pre_election: true
I20260809 15:50:20.446583 9962 leader_election.cc:288] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.443778 9918 raft_consensus.cc:348] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.447399 9918 raft_consensus.cc:374] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.447698 9918 raft_consensus.cc:729] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.446589 9661 leader_election.cc:302] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, e54dcbceb00649eb944818025b5a969f; no voters:
I20260809 15:50:20.449311 9962 raft_consensus.cc:2781] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260809 15:50:20.449435 9661 leader_election.cc:341] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:20.449762 9962 raft_consensus.cc:482] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:50:20.450167 9962 raft_consensus.cc:3037] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.450480 9923 raft_consensus.cc:482] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:20.451198 9918 consensus_queue.cc:260] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.450960 9923 raft_consensus.cc:504] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.454773 9923 leader_election.cc:288] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.456326 9703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9d11b0e8c644e089dd34200345495c7" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "042b068763ff48718439228aebb5cfcb" is_pre_election: true
I20260809 15:50:20.456988 9874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9d11b0e8c644e089dd34200345495c7" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" is_pre_election: true
I20260809 15:50:20.457568 9703 raft_consensus.cc:2379] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e54dcbceb00649eb944818025b5a969f in current term 1: Already voted for candidate 4a0709a93f76476a8acbfa4e9eb955d7 in this term.
I20260809 15:50:20.448988 9704 raft_consensus.cc:2454] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a0709a93f76476a8acbfa4e9eb955d7 in term 1.
I20260809 15:50:20.462795 9918 ts_tablet_manager.cc:1428] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f: Time spent starting tablet: real 0.033s user 0.007s sys 0.003s
I20260809 15:50:20.463435 9918 tablet_bootstrap.cc:492] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f: Bootstrap starting.
I20260809 15:50:20.461458 9747 leader_election.cc:302] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e54dcbceb00649eb944818025b5a969f; no voters: 042b068763ff48718439228aebb5cfcb, 4a0709a93f76476a8acbfa4e9eb955d7
I20260809 15:50:20.465541 9923 raft_consensus.cc:3037] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.467249 9830 leader_election.cc:302] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, 4a0709a93f76476a8acbfa4e9eb955d7; no voters: e54dcbceb00649eb944818025b5a969f
I20260809 15:50:20.472558 9945 raft_consensus.cc:2781] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:20.474282 9945 raft_consensus.cc:686] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 LEADER]: Becoming Leader. State: Replica: 4a0709a93f76476a8acbfa4e9eb955d7, State: Running, Role: LEADER
I20260809 15:50:20.475178 9945 consensus_queue.cc:237] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.478197 9962 raft_consensus.cc:504] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } }
I20260809 15:50:20.481156 9962 leader_election.cc:288] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 election: Requested vote from peers e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.481355 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e74d7900d9d485092a9e8b144fca6b0" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e54dcbceb00649eb944818025b5a969f"
I20260809 15:50:20.484211 9874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e74d7900d9d485092a9e8b144fca6b0" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
I20260809 15:50:20.479661 9585 catalog_manager.cc:5495] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb reported cstate change: term changed from 0 to 1, leader changed from <none> to 042b068763ff48718439228aebb5cfcb (127.9.84.1). New cstate: current_term: 1 leader_uuid: "042b068763ff48718439228aebb5cfcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: UNKNOWN } } }
W20260809 15:50:20.486649 9661 leader_election.cc:341] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:20.485083 9788 raft_consensus.cc:3037] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.490432 9923 raft_consensus.cc:2726] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:50:20.494148 9921 tablet_bootstrap.cc:492] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7: No bootstrap required, opened a new log
I20260809 15:50:20.496050 9921 ts_tablet_manager.cc:1397] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7: Time spent bootstrapping tablet: real 0.086s user 0.030s sys 0.026s
I20260809 15:50:20.494369 9788 raft_consensus.cc:2454] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 042b068763ff48718439228aebb5cfcb in term 1.
I20260809 15:50:20.497169 9918 tablet_bootstrap.cc:654] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.495452 9923 raft_consensus.cc:482] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:20.502907 9661 leader_election.cc:302] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, e54dcbceb00649eb944818025b5a969f; no voters: 4a0709a93f76476a8acbfa4e9eb955d7
I20260809 15:50:20.500116 9923 raft_consensus.cc:504] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
W20260809 15:50:20.507697 9747 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:35125, user_credentials={real_user=slave}} blocked reactor thread for 46374.6us
I20260809 15:50:20.510954 9704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7526b699d7d4b9680c8f2e98b9c72e2" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "042b068763ff48718439228aebb5cfcb" is_pre_election: true
I20260809 15:50:20.511201 9921 raft_consensus.cc:348] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } }
I20260809 15:50:20.511596 9704 raft_consensus.cc:2454] T b7526b699d7d4b9680c8f2e98b9c72e2 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 0.
I20260809 15:50:20.511338 9874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7526b699d7d4b9680c8f2e98b9c72e2" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" is_pre_election: true
I20260809 15:50:20.512288 9874 raft_consensus.cc:2454] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 0.
I20260809 15:50:20.513448 9747 leader_election.cc:302] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, e54dcbceb00649eb944818025b5a969f; no voters:
I20260809 15:50:20.514249 9923 leader_election.cc:288] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.505940 9962 raft_consensus.cc:2781] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:20.511933 9921 raft_consensus.cc:374] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.508240 9584 catalog_manager.cc:5495] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3). New cstate: current_term: 1 leader_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: HEALTHY } } }
I20260809 15:50:20.515411 9962 raft_consensus.cc:686] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 1 LEADER]: Becoming Leader. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Running, Role: LEADER
I20260809 15:50:20.516166 9921 raft_consensus.cc:729] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a0709a93f76476a8acbfa4e9eb955d7, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.515162 9923 raft_consensus.cc:2781] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:50:20.517673 9962 consensus_queue.cc:237] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } }
I20260809 15:50:20.517393 9921 consensus_queue.cc:260] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } }
I20260809 15:50:20.520025 9918 tablet_bootstrap.cc:492] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f: No bootstrap required, opened a new log
I20260809 15:50:20.518548 9923 raft_consensus.cc:482] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:50:20.520411 9923 raft_consensus.cc:3037] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.521806 9921 ts_tablet_manager.cc:1428] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7: Time spent starting tablet: real 0.025s user 0.005s sys 0.002s
I20260809 15:50:20.522914 9921 tablet_bootstrap.cc:492] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7: Bootstrap starting.
I20260809 15:50:20.523108 9918 ts_tablet_manager.cc:1397] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f: Time spent bootstrapping tablet: real 0.060s user 0.015s sys 0.006s
I20260809 15:50:20.540962 9968 raft_consensus.cc:482] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:20.542764 9968 raft_consensus.cc:504] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.543262 9918 raft_consensus.cc:348] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.544373 9918 raft_consensus.cc:374] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.544697 9918 raft_consensus.cc:729] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.545547 9918 consensus_queue.cc:260] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.548161 9918 ts_tablet_manager.cc:1428] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f: Time spent starting tablet: real 0.025s user 0.011s sys 0.007s
I20260809 15:50:20.549791 9923 raft_consensus.cc:504] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.551401 9968 leader_election.cc:288] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.551875 9923 leader_election.cc:288] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 election: Requested vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.552834 9874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c5ba0a4aea4aebaeecf735050a4528" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" is_pre_election: true
I20260809 15:50:20.554001 9873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7526b699d7d4b9680c8f2e98b9c72e2" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
I20260809 15:50:20.554594 9873 raft_consensus.cc:3037] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.554463 9704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c5ba0a4aea4aebaeecf735050a4528" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "042b068763ff48718439228aebb5cfcb" is_pre_election: true
I20260809 15:50:20.555101 9704 raft_consensus.cc:2454] T 90c5ba0a4aea4aebaeecf735050a4528 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 0.
I20260809 15:50:20.556879 9921 tablet_bootstrap.cc:654] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.560906 9747 leader_election.cc:302] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, e54dcbceb00649eb944818025b5a969f; no voters:
I20260809 15:50:20.561084 9945 raft_consensus.cc:482] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:20.561704 9923 raft_consensus.cc:2781] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:50:20.561612 9945 raft_consensus.cc:504] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } }
I20260809 15:50:20.562197 9923 raft_consensus.cc:482] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 15:50:20.562057 9749 leader_election.cc:341] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:20.562709 9923 raft_consensus.cc:3037] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.563846 9945 leader_election.cc:288] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899), 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125)
I20260809 15:50:20.562237 9704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7526b699d7d4b9680c8f2e98b9c72e2" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "042b068763ff48718439228aebb5cfcb"
I20260809 15:50:20.567206 9703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e74d7900d9d485092a9e8b144fca6b0" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "042b068763ff48718439228aebb5cfcb" is_pre_election: true
I20260809 15:50:20.567672 9873 raft_consensus.cc:2454] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 1.
I20260809 15:50:20.567864 9704 raft_consensus.cc:3037] T b7526b699d7d4b9680c8f2e98b9c72e2 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.567970 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e74d7900d9d485092a9e8b144fca6b0" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e54dcbceb00649eb944818025b5a969f" is_pre_election: true
I20260809 15:50:20.568876 9788 raft_consensus.cc:2379] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4a0709a93f76476a8acbfa4e9eb955d7 in current term 1: Already voted for candidate 042b068763ff48718439228aebb5cfcb in this term.
I20260809 15:50:20.568742 9749 leader_election.cc:302] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a0709a93f76476a8acbfa4e9eb955d7, e54dcbceb00649eb944818025b5a969f; no voters:
I20260809 15:50:20.569497 9968 raft_consensus.cc:2781] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:20.570006 9968 raft_consensus.cc:686] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [term 1 LEADER]: Becoming Leader. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Running, Role: LEADER
I20260809 15:50:20.570161 9832 leader_election.cc:302] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4a0709a93f76476a8acbfa4e9eb955d7; no voters: 042b068763ff48718439228aebb5cfcb, e54dcbceb00649eb944818025b5a969f
I20260809 15:50:20.570971 9945 raft_consensus.cc:3037] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.570937 9968 consensus_queue.cc:237] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.573529 9585 catalog_manager.cc:5495] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb reported cstate change: term changed from 0 to 1, leader changed from <none> to 042b068763ff48718439228aebb5cfcb (127.9.84.1). New cstate: current_term: 1 leader_uuid: "042b068763ff48718439228aebb5cfcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: HEALTHY } } }
I20260809 15:50:20.577491 9945 raft_consensus.cc:2726] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:50:20.577472 9704 raft_consensus.cc:2454] T b7526b699d7d4b9680c8f2e98b9c72e2 P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 1.
I20260809 15:50:20.583143 9923 raft_consensus.cc:504] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.585628 9923 leader_election.cc:288] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 election: Requested vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.586443 9704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c5ba0a4aea4aebaeecf735050a4528" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "042b068763ff48718439228aebb5cfcb"
I20260809 15:50:20.586700 9873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c5ba0a4aea4aebaeecf735050a4528" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
I20260809 15:50:20.587075 9704 raft_consensus.cc:3037] T 90c5ba0a4aea4aebaeecf735050a4528 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:50:20.587836 9749 leader_election.cc:341] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:20.592293 9704 raft_consensus.cc:2454] T 90c5ba0a4aea4aebaeecf735050a4528 P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 1.
I20260809 15:50:20.592819 9921 tablet_bootstrap.cc:492] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7: No bootstrap required, opened a new log
I20260809 15:50:20.598747 9968 raft_consensus.cc:482] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:20.593199 9747 leader_election.cc:302] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, e54dcbceb00649eb944818025b5a969f; no voters: 4a0709a93f76476a8acbfa4e9eb955d7
I20260809 15:50:20.600013 9921 ts_tablet_manager.cc:1397] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7: Time spent bootstrapping tablet: real 0.077s user 0.015s sys 0.025s
I20260809 15:50:20.599997 9968 raft_consensus.cc:504] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.600898 9923 raft_consensus.cc:2781] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:20.601523 9923 raft_consensus.cc:686] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [term 1 LEADER]: Becoming Leader. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Running, Role: LEADER
I20260809 15:50:20.602226 9968 leader_election.cc:288] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.602334 9923 consensus_queue.cc:237] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.603207 9921 raft_consensus.cc:348] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.603899 9921 raft_consensus.cc:374] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.604202 9921 raft_consensus.cc:729] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a0709a93f76476a8acbfa4e9eb955d7, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.592842 9585 catalog_manager.cc:5495] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f reported cstate change: term changed from 0 to 1, leader changed from <none> to e54dcbceb00649eb944818025b5a969f (127.9.84.2). New cstate: current_term: 1 leader_uuid: "e54dcbceb00649eb944818025b5a969f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:20.604924 9704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "726df19d010d4f008eb75311543d0dd6" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "042b068763ff48718439228aebb5cfcb" is_pre_election: true
I20260809 15:50:20.604802 9921 consensus_queue.cc:260] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.605633 9704 raft_consensus.cc:2454] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 0.
I20260809 15:50:20.607105 9921 ts_tablet_manager.cc:1428] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260809 15:50:20.607055 9873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "726df19d010d4f008eb75311543d0dd6" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" is_pre_election: true
I20260809 15:50:20.607841 9873 raft_consensus.cc:2454] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 0.
I20260809 15:50:20.608179 9921 tablet_bootstrap.cc:492] T 90c5ba0a4aea4aebaeecf735050a4528 P 4a0709a93f76476a8acbfa4e9eb955d7: Bootstrap starting.
I20260809 15:50:20.608916 9749 leader_election.cc:302] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a0709a93f76476a8acbfa4e9eb955d7, e54dcbceb00649eb944818025b5a969f; no voters:
I20260809 15:50:20.609777 9968 raft_consensus.cc:2781] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:50:20.610178 9968 raft_consensus.cc:482] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:50:20.610543 9968 raft_consensus.cc:3037] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.614166 9921 tablet_bootstrap.cc:654] T 90c5ba0a4aea4aebaeecf735050a4528 P 4a0709a93f76476a8acbfa4e9eb955d7: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:20.618587 9968 raft_consensus.cc:504] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.621590 9704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "726df19d010d4f008eb75311543d0dd6" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "042b068763ff48718439228aebb5cfcb"
I20260809 15:50:20.622236 9704 raft_consensus.cc:3037] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.626667 9972 raft_consensus.cc:482] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:20.627182 9972 raft_consensus.cc:504] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.629297 9972 leader_election.cc:288] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.631436 9921 tablet_bootstrap.cc:492] T 90c5ba0a4aea4aebaeecf735050a4528 P 4a0709a93f76476a8acbfa4e9eb955d7: No bootstrap required, opened a new log
I20260809 15:50:20.632126 9921 ts_tablet_manager.cc:1397] T 90c5ba0a4aea4aebaeecf735050a4528 P 4a0709a93f76476a8acbfa4e9eb955d7: Time spent bootstrapping tablet: real 0.024s user 0.013s sys 0.004s
I20260809 15:50:20.636261 9921 raft_consensus.cc:348] T 90c5ba0a4aea4aebaeecf735050a4528 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.637004 9921 raft_consensus.cc:374] T 90c5ba0a4aea4aebaeecf735050a4528 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:20.636262 9945 consensus_queue.cc:1035] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:50:20.637286 9921 raft_consensus.cc:729] T 90c5ba0a4aea4aebaeecf735050a4528 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a0709a93f76476a8acbfa4e9eb955d7, State: Initialized, Role: FOLLOWER
I20260809 15:50:20.638757 9585 catalog_manager.cc:5495] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f reported cstate change: term changed from 0 to 1, leader changed from <none> to e54dcbceb00649eb944818025b5a969f (127.9.84.2). New cstate: current_term: 1 leader_uuid: "e54dcbceb00649eb944818025b5a969f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:20.637897 9921 consensus_queue.cc:260] T 90c5ba0a4aea4aebaeecf735050a4528 P 4a0709a93f76476a8acbfa4e9eb955d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.640389 9703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10055574bc224746bde901e394facf6a" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "042b068763ff48718439228aebb5cfcb" is_pre_election: true
I20260809 15:50:20.642635 9703 raft_consensus.cc:2454] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 0.
I20260809 15:50:20.643179 9962 raft_consensus.cc:482] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:20.643893 9873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10055574bc224746bde901e394facf6a" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" is_pre_election: true
I20260809 15:50:20.643757 9962 raft_consensus.cc:504] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.644500 9873 raft_consensus.cc:2454] T 10055574bc224746bde901e394facf6a P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 0.
I20260809 15:50:20.645727 9962 leader_election.cc:288] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.646167 9921 ts_tablet_manager.cc:1428] T 90c5ba0a4aea4aebaeecf735050a4528 P 4a0709a93f76476a8acbfa4e9eb955d7: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260809 15:50:20.647977 9873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10055574bc224746bde901e394facf6a" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" is_pre_election: true
I20260809 15:50:20.648118 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10055574bc224746bde901e394facf6a" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e54dcbceb00649eb944818025b5a969f" is_pre_election: true
I20260809 15:50:20.648382 9873 raft_consensus.cc:2454] T 10055574bc224746bde901e394facf6a P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 042b068763ff48718439228aebb5cfcb in term 0.
I20260809 15:50:20.648805 9788 raft_consensus.cc:2454] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 042b068763ff48718439228aebb5cfcb in term 0.
I20260809 15:50:20.649510 9661 leader_election.cc:302] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, 4a0709a93f76476a8acbfa4e9eb955d7; no voters:
I20260809 15:50:20.650537 9933 raft_consensus.cc:2781] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:50:20.650452 9749 leader_election.cc:302] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a0709a93f76476a8acbfa4e9eb955d7, e54dcbceb00649eb944818025b5a969f; no voters:
I20260809 15:50:20.650983 9933 raft_consensus.cc:482] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:50:20.651291 9933 raft_consensus.cc:3037] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.651331 9923 raft_consensus.cc:2781] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:50:20.651736 9923 raft_consensus.cc:482] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:50:20.652036 9923 raft_consensus.cc:3037] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.658113 9968 leader_election.cc:288] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 election: Requested vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.659610 9873 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "726df19d010d4f008eb75311543d0dd6" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
I20260809 15:50:20.660218 9873 raft_consensus.cc:3037] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.655076 9704 raft_consensus.cc:2454] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 1.
I20260809 15:50:20.666605 9933 raft_consensus.cc:504] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.664119 9747 leader_election.cc:302] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, e54dcbceb00649eb944818025b5a969f; no voters:
I20260809 15:50:20.662434 9923 raft_consensus.cc:504] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.671350 9968 raft_consensus.cc:2781] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:20.672009 9968 raft_consensus.cc:686] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 1 LEADER]: Becoming Leader. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Running, Role: LEADER
I20260809 15:50:20.672866 9968 consensus_queue.cc:237] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.675896 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10055574bc224746bde901e394facf6a" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e54dcbceb00649eb944818025b5a969f"
I20260809 15:50:20.676736 9788 raft_consensus.cc:2379] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 042b068763ff48718439228aebb5cfcb in current term 1: Already voted for candidate e54dcbceb00649eb944818025b5a969f in this term.
I20260809 15:50:20.679189 9933 leader_election.cc:288] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 election: Requested vote from peers e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.681584 9931 consensus_queue.cc:1035] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:50:20.681325 9701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10055574bc224746bde901e394facf6a" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "042b068763ff48718439228aebb5cfcb"
I20260809 15:50:20.684845 9933 raft_consensus.cc:482] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:20.685318 9933 raft_consensus.cc:504] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.687477 9701 raft_consensus.cc:2379] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate e54dcbceb00649eb944818025b5a969f in current term 1: Already voted for candidate 042b068763ff48718439228aebb5cfcb in this term.
I20260809 15:50:20.687538 9933 leader_election.cc:288] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.689865 9874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10055574bc224746bde901e394facf6a" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
I20260809 15:50:20.690613 9874 raft_consensus.cc:3037] T 10055574bc224746bde901e394facf6a P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.692278 9923 leader_election.cc:288] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 election: Requested vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.692344 9872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fa161c687aa4ee390cca51e913fe50f" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" is_pre_election: true
I20260809 15:50:20.693091 9872 raft_consensus.cc:2454] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 042b068763ff48718439228aebb5cfcb in term 0.
I20260809 15:50:20.683430 9873 raft_consensus.cc:2454] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 1.
I20260809 15:50:20.696617 9871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10055574bc224746bde901e394facf6a" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
I20260809 15:50:20.701037 9584 catalog_manager.cc:5495] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f reported cstate change: term changed from 0 to 1, leader changed from <none> to e54dcbceb00649eb944818025b5a969f (127.9.84.2). New cstate: current_term: 1 leader_uuid: "e54dcbceb00649eb944818025b5a969f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:20.703307 9749 leader_election.cc:302] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e54dcbceb00649eb944818025b5a969f; no voters: 042b068763ff48718439228aebb5cfcb, 4a0709a93f76476a8acbfa4e9eb955d7
I20260809 15:50:20.704329 9968 raft_consensus.cc:2726] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 15:50:20.698645 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fa161c687aa4ee390cca51e913fe50f" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e54dcbceb00649eb944818025b5a969f" is_pre_election: true
I20260809 15:50:20.705880 9788 raft_consensus.cc:2454] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 042b068763ff48718439228aebb5cfcb in term 0.
I20260809 15:50:20.707577 9874 raft_consensus.cc:2454] T 10055574bc224746bde901e394facf6a P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 042b068763ff48718439228aebb5cfcb in term 1.
I20260809 15:50:20.708284 9661 leader_election.cc:302] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, 4a0709a93f76476a8acbfa4e9eb955d7; no voters:
I20260809 15:50:20.709830 9661 leader_election.cc:302] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, 4a0709a93f76476a8acbfa4e9eb955d7; no voters: e54dcbceb00649eb944818025b5a969f
I20260809 15:50:20.710255 9933 raft_consensus.cc:2781] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:50:20.710726 9933 raft_consensus.cc:482] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:50:20.710942 9962 raft_consensus.cc:2781] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:20.711175 9933 raft_consensus.cc:3037] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.711504 9962 raft_consensus.cc:686] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 1 LEADER]: Becoming Leader. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Running, Role: LEADER
I20260809 15:50:20.712419 9962 consensus_queue.cc:237] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.722303 9933 raft_consensus.cc:504] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.724587 9933 leader_election.cc:288] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 election: Requested vote from peers e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.725975 9874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fa161c687aa4ee390cca51e913fe50f" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
I20260809 15:50:20.726682 9874 raft_consensus.cc:3037] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.726825 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fa161c687aa4ee390cca51e913fe50f" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e54dcbceb00649eb944818025b5a969f"
I20260809 15:50:20.727429 9788 raft_consensus.cc:3037] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.725201 9585 catalog_manager.cc:5495] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb reported cstate change: term changed from 0 to 1, leader changed from <none> to 042b068763ff48718439228aebb5cfcb (127.9.84.1). New cstate: current_term: 1 leader_uuid: "042b068763ff48718439228aebb5cfcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:20.734601 9788 raft_consensus.cc:2454] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 042b068763ff48718439228aebb5cfcb in term 1.
I20260809 15:50:20.736224 9661 leader_election.cc:302] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, e54dcbceb00649eb944818025b5a969f; no voters:
I20260809 15:50:20.736382 9874 raft_consensus.cc:2454] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 042b068763ff48718439228aebb5cfcb in term 1.
I20260809 15:50:20.737201 9933 raft_consensus.cc:2781] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:20.738344 9933 raft_consensus.cc:686] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 1 LEADER]: Becoming Leader. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Running, Role: LEADER
I20260809 15:50:20.739243 9933 consensus_queue.cc:237] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.747494 9585 catalog_manager.cc:5495] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb reported cstate change: term changed from 0 to 1, leader changed from <none> to 042b068763ff48718439228aebb5cfcb (127.9.84.1). New cstate: current_term: 1 leader_uuid: "042b068763ff48718439228aebb5cfcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: UNKNOWN } } }
W20260809 15:50:20.753793 9641 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 15:50:20.784236 9788 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "726df19d010d4f008eb75311543d0dd6"
dest_uuid: "e54dcbceb00649eb944818025b5a969f"
mode: GRACEFUL
new_leader_uuid: "042b068763ff48718439228aebb5cfcb"
from {username='slave'} at 127.0.0.1:43754
I20260809 15:50:20.784996 9788 raft_consensus.cc:595] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 1 LEADER]: Received request to transfer leadership to TS 042b068763ff48718439228aebb5cfcb
I20260809 15:50:20.804098 9874 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a9d11b0e8c644e089dd34200345495c7"
dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
mode: GRACEFUL
new_leader_uuid: "e54dcbceb00649eb944818025b5a969f"
from {username='slave'} at 127.0.0.1:41180
I20260809 15:50:20.804615 9874 raft_consensus.cc:595] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 LEADER]: Received request to transfer leadership to TS e54dcbceb00649eb944818025b5a969f
I20260809 15:50:20.829187 9704 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d9963b7dc24a41838578644fa0572566"
dest_uuid: "042b068763ff48718439228aebb5cfcb"
mode: GRACEFUL
new_leader_uuid: "e54dcbceb00649eb944818025b5a969f"
from {username='slave'} at 127.0.0.1:40548
I20260809 15:50:20.829727 9704 raft_consensus.cc:595] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 1 LEADER]: Received request to transfer leadership to TS e54dcbceb00649eb944818025b5a969f
I20260809 15:50:20.834430 9874 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "691afd4416c84d7d8e595a5b7c5e7d4b"
dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
mode: GRACEFUL
new_leader_uuid: "e54dcbceb00649eb944818025b5a969f"
from {username='slave'} at 127.0.0.1:41180
I20260809 15:50:20.834913 9874 raft_consensus.cc:595] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 LEADER]: Received request to transfer leadership to TS e54dcbceb00649eb944818025b5a969f
I20260809 15:50:20.839285 9788 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b7526b699d7d4b9680c8f2e98b9c72e2"
dest_uuid: "e54dcbceb00649eb944818025b5a969f"
mode: GRACEFUL
new_leader_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
from {username='slave'} at 127.0.0.1:43754
I20260809 15:50:20.839893 9788 raft_consensus.cc:595] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [term 1 LEADER]: Received request to transfer leadership to TS 4a0709a93f76476a8acbfa4e9eb955d7
I20260809 15:50:20.863412 9933 consensus_queue.cc:1035] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:20.870244 9962 raft_consensus.cc:979] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb: : Instructing follower e54dcbceb00649eb944818025b5a969f to start an election
I20260809 15:50:20.870752 9962 raft_consensus.cc:1067] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 1 LEADER]: Signalling peer e54dcbceb00649eb944818025b5a969f to start an election
I20260809 15:50:20.875027 9787 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d9963b7dc24a41838578644fa0572566"
dest_uuid: "e54dcbceb00649eb944818025b5a969f"
from {username='slave'} at 127.0.0.1:43750
I20260809 15:50:20.876019 9787 raft_consensus.cc:482] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:20.876443 9787 raft_consensus.cc:3037] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:20.877532 9962 consensus_queue.cc:1035] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:20.883445 9968 raft_consensus.cc:482] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:20.883352 9787 raft_consensus.cc:504] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.884086 9968 raft_consensus.cc:504] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.887876 9871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691afd4416c84d7d8e595a5b7c5e7d4b" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" is_pre_election: true
I20260809 15:50:20.889755 9787 leader_election.cc:288] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 2 election: Requested vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.892433 9701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9963b7dc24a41838578644fa0572566" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "042b068763ff48718439228aebb5cfcb"
I20260809 15:50:20.893033 9701 raft_consensus.cc:3032] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:50:20.893378 9701 raft_consensus.cc:729] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Running, Role: LEADER
I20260809 15:50:20.894330 9945 raft_consensus.cc:482] T 90c5ba0a4aea4aebaeecf735050a4528 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:20.894203 9701 consensus_queue.cc:260] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.894763 9945 raft_consensus.cc:504] T 90c5ba0a4aea4aebaeecf735050a4528 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.895495 9701 raft_consensus.cc:3037] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:20.896720 9945 leader_election.cc:288] T 90c5ba0a4aea4aebaeecf735050a4528 P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899)
I20260809 15:50:20.897415 9703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c5ba0a4aea4aebaeecf735050a4528" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "042b068763ff48718439228aebb5cfcb" is_pre_election: true
I20260809 15:50:20.897874 9704 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691afd4416c84d7d8e595a5b7c5e7d4b" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "042b068763ff48718439228aebb5cfcb" is_pre_election: true
I20260809 15:50:20.898187 9703 raft_consensus.cc:2379] T 90c5ba0a4aea4aebaeecf735050a4528 P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4a0709a93f76476a8acbfa4e9eb955d7 in current term 1: Already voted for candidate e54dcbceb00649eb944818025b5a969f in this term.
I20260809 15:50:20.898663 9704 raft_consensus.cc:2379] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e54dcbceb00649eb944818025b5a969f in current term 1: Already voted for candidate 4a0709a93f76476a8acbfa4e9eb955d7 in this term.
I20260809 15:50:20.899549 9871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9963b7dc24a41838578644fa0572566" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
I20260809 15:50:20.900092 9747 leader_election.cc:302] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e54dcbceb00649eb944818025b5a969f; no voters: 042b068763ff48718439228aebb5cfcb, 4a0709a93f76476a8acbfa4e9eb955d7
I20260809 15:50:20.900482 9968 leader_election.cc:288] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:20.901209 9968 raft_consensus.cc:3037] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.902058 9701 raft_consensus.cc:2454] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 2.
I20260809 15:50:20.904865 9747 leader_election.cc:302] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, e54dcbceb00649eb944818025b5a969f; no voters: 4a0709a93f76476a8acbfa4e9eb955d7
I20260809 15:50:20.905683 9923 raft_consensus.cc:2781] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:50:20.906142 9923 raft_consensus.cc:686] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Becoming Leader. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Running, Role: LEADER
I20260809 15:50:20.906911 9923 consensus_queue.cc:237] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:20.907401 9968 raft_consensus.cc:2726] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:50:20.902812 9787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "90c5ba0a4aea4aebaeecf735050a4528" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e54dcbceb00649eb944818025b5a969f" is_pre_election: true
I20260809 15:50:20.910022 9832 leader_election.cc:302] T 90c5ba0a4aea4aebaeecf735050a4528 P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4a0709a93f76476a8acbfa4e9eb955d7; no voters: 042b068763ff48718439228aebb5cfcb, e54dcbceb00649eb944818025b5a969f
I20260809 15:50:20.910861 9931 raft_consensus.cc:3037] T 90c5ba0a4aea4aebaeecf735050a4528 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:20.915012 9931 raft_consensus.cc:2726] T 90c5ba0a4aea4aebaeecf735050a4528 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:50:20.914698 9584 catalog_manager.cc:5495] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f reported cstate change: term changed from 1 to 2, leader changed from 042b068763ff48718439228aebb5cfcb (127.9.84.1) to e54dcbceb00649eb944818025b5a969f (127.9.84.2). New cstate: current_term: 2 leader_uuid: "e54dcbceb00649eb944818025b5a969f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:20.939826 9962 consensus_queue.cc:1035] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:20.950842 9933 consensus_queue.cc:1035] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:21.014000 9931 consensus_queue.cc:1035] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:50:21.034870 9999 consensus_queue.cc:1035] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:21.049782 9998 raft_consensus.cc:979] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7: : Instructing follower e54dcbceb00649eb944818025b5a969f to start an election
I20260809 15:50:21.050220 9931 raft_consensus.cc:1067] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 LEADER]: Signalling peer e54dcbceb00649eb944818025b5a969f to start an election
I20260809 15:50:21.051762 9788 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "691afd4416c84d7d8e595a5b7c5e7d4b"
dest_uuid: "e54dcbceb00649eb944818025b5a969f"
from {username='slave'} at 127.0.0.1:43748
I20260809 15:50:21.052290 9788 raft_consensus.cc:482] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:21.052563 9788 raft_consensus.cc:3037] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:21.057482 9788 raft_consensus.cc:504] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:21.059376 9788 leader_election.cc:288] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 2 election: Requested vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:21.060180 9701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691afd4416c84d7d8e595a5b7c5e7d4b" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "042b068763ff48718439228aebb5cfcb"
I20260809 15:50:21.060396 9874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691afd4416c84d7d8e595a5b7c5e7d4b" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
I20260809 15:50:21.060797 9701 raft_consensus.cc:3037] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:21.061005 9874 raft_consensus.cc:3032] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:50:21.061260 9874 raft_consensus.cc:729] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4a0709a93f76476a8acbfa4e9eb955d7, State: Running, Role: LEADER
I20260809 15:50:21.061838 9874 consensus_queue.cc:260] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:21.062886 9874 raft_consensus.cc:3037] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:21.067497 9874 raft_consensus.cc:2454] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 2.
I20260809 15:50:21.068590 9749 leader_election.cc:302] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a0709a93f76476a8acbfa4e9eb955d7, e54dcbceb00649eb944818025b5a969f; no voters:
I20260809 15:50:21.069383 9923 raft_consensus.cc:2781] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:50:21.069849 9923 raft_consensus.cc:686] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Becoming Leader. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Running, Role: LEADER
I20260809 15:50:21.070981 9701 raft_consensus.cc:2454] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 2.
I20260809 15:50:21.070688 9923 consensus_queue.cc:237] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:21.074860 9923 consensus_queue.cc:1035] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:21.080086 9584 catalog_manager.cc:5495] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f reported cstate change: term changed from 1 to 2, leader changed from 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3) to e54dcbceb00649eb944818025b5a969f (127.9.84.2). New cstate: current_term: 2 leader_uuid: "e54dcbceb00649eb944818025b5a969f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:21.081297 9923 consensus_queue.cc:1035] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:21.089056 9972 raft_consensus.cc:979] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f: : Instructing follower 042b068763ff48718439228aebb5cfcb to start an election
I20260809 15:50:21.089478 9972 raft_consensus.cc:1067] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 1 LEADER]: Signalling peer 042b068763ff48718439228aebb5cfcb to start an election
I20260809 15:50:21.092096 9703 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "726df19d010d4f008eb75311543d0dd6"
dest_uuid: "042b068763ff48718439228aebb5cfcb"
from {username='slave'} at 127.0.0.1:40520
I20260809 15:50:21.092649 9703 raft_consensus.cc:482] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:21.093003 9703 raft_consensus.cc:3037] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:21.095058 9968 consensus_queue.cc:1035] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:21.102905 9703 raft_consensus.cc:504] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:21.104003 9704 raft_consensus.cc:1226] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Rejecting Update request from peer e54dcbceb00649eb944818025b5a969f for earlier term 1. Current term is 2. Ops: []
I20260809 15:50:21.104241 9972 consensus_queue.cc:1035] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:21.109817 9933 consensus_queue.cc:1035] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:21.110112 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "726df19d010d4f008eb75311543d0dd6" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e54dcbceb00649eb944818025b5a969f"
I20260809 15:50:21.110721 9788 raft_consensus.cc:3032] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:50:21.111023 9788 raft_consensus.cc:729] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Running, Role: LEADER
I20260809 15:50:21.111748 9788 consensus_queue.cc:260] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:21.112717 9788 raft_consensus.cc:3037] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Advancing to term 2
W20260809 15:50:21.113541 9968 consensus_queue.cc:1159] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "042b068763ff48718439228aebb5cfcb" responder_term: 2 status { last_received { term: 1 index: 1 } last_committed_idx: 0 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer e54dcbceb00649eb944818025b5a969f for earlier term 1. Current term is 2. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260809 15:50:21.121726 9703 leader_election.cc:288] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 2 election: Requested vote from peers e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:21.123544 9872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "726df19d010d4f008eb75311543d0dd6" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
I20260809 15:50:21.130106 9999 raft_consensus.cc:979] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7: : Instructing follower e54dcbceb00649eb944818025b5a969f to start an election
I20260809 15:50:21.132200 9999 raft_consensus.cc:1067] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 LEADER]: Signalling peer e54dcbceb00649eb944818025b5a969f to start an election
I20260809 15:50:21.134285 9785 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a9d11b0e8c644e089dd34200345495c7"
dest_uuid: "e54dcbceb00649eb944818025b5a969f"
from {username='slave'} at 127.0.0.1:43748
I20260809 15:50:21.134963 9785 raft_consensus.cc:482] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:21.135432 9785 raft_consensus.cc:3037] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:21.153280 9785 raft_consensus.cc:504] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:21.135442 9788 raft_consensus.cc:2454] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 042b068763ff48718439228aebb5cfcb in term 2.
I20260809 15:50:21.155205 9785 leader_election.cc:288] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 2 election: Requested vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:21.155931 9661 leader_election.cc:302] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, e54dcbceb00649eb944818025b5a969f; no voters: 4a0709a93f76476a8acbfa4e9eb955d7
I20260809 15:50:21.157734 9933 raft_consensus.cc:2781] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:50:21.158221 9933 raft_consensus.cc:686] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 2 LEADER]: Becoming Leader. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Running, Role: LEADER
I20260809 15:50:21.159020 9874 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9d11b0e8c644e089dd34200345495c7" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
I20260809 15:50:21.159624 9874 raft_consensus.cc:3032] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:50:21.159993 9874 raft_consensus.cc:729] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4a0709a93f76476a8acbfa4e9eb955d7, State: Running, Role: LEADER
I20260809 15:50:21.159041 9933 consensus_queue.cc:237] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:21.159943 9929 consensus_queue.cc:1035] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:50:21.160940 9874 consensus_queue.cc:260] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:21.162055 9874 raft_consensus.cc:3037] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:21.162190 9968 consensus_queue.cc:1035] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260809 15:50:21.168854 9972 consensus_queue.cc:1035] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:21.163601 9703 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9d11b0e8c644e089dd34200345495c7" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "042b068763ff48718439228aebb5cfcb"
I20260809 15:50:21.171072 9703 raft_consensus.cc:3037] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:21.176123 9584 catalog_manager.cc:5495] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb reported cstate change: term changed from 1 to 2, leader changed from e54dcbceb00649eb944818025b5a969f (127.9.84.2) to 042b068763ff48718439228aebb5cfcb (127.9.84.1). New cstate: current_term: 2 leader_uuid: "042b068763ff48718439228aebb5cfcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:21.182325 9703 raft_consensus.cc:2454] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 2.
I20260809 15:50:21.186403 9747 leader_election.cc:302] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, e54dcbceb00649eb944818025b5a969f; no voters:
I20260809 15:50:21.187211 9972 raft_consensus.cc:2781] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:50:21.187423 9874 raft_consensus.cc:2454] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 2.
I20260809 15:50:21.188397 9972 raft_consensus.cc:686] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Becoming Leader. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Running, Role: LEADER
I20260809 15:50:21.189232 9972 consensus_queue.cc:237] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:21.200168 9584 catalog_manager.cc:5495] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f reported cstate change: term changed from 1 to 2, leader changed from 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3) to e54dcbceb00649eb944818025b5a969f (127.9.84.2). New cstate: current_term: 2 leader_uuid: "e54dcbceb00649eb944818025b5a969f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:21.201442 9968 raft_consensus.cc:979] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f: : Instructing follower 4a0709a93f76476a8acbfa4e9eb955d7 to start an election
I20260809 15:50:21.201828 9968 raft_consensus.cc:1067] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [term 1 LEADER]: Signalling peer 4a0709a93f76476a8acbfa4e9eb955d7 to start an election
I20260809 15:50:21.211481 9871 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b7526b699d7d4b9680c8f2e98b9c72e2"
dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
from {username='slave'} at 127.0.0.1:41158
I20260809 15:50:21.212025 9871 raft_consensus.cc:482] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:21.212407 9871 raft_consensus.cc:3037] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:21.219045 9871 raft_consensus.cc:504] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:21.221071 9871 leader_election.cc:288] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 2 election: Requested vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899)
I20260809 15:50:21.221972 9701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7526b699d7d4b9680c8f2e98b9c72e2" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "042b068763ff48718439228aebb5cfcb"
I20260809 15:50:21.222574 9701 raft_consensus.cc:3037] T b7526b699d7d4b9680c8f2e98b9c72e2 P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:21.222754 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7526b699d7d4b9680c8f2e98b9c72e2" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e54dcbceb00649eb944818025b5a969f"
I20260809 15:50:21.223407 9788 raft_consensus.cc:3032] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:50:21.223664 9788 raft_consensus.cc:729] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [term 1 LEADER]: Becoming Follower/Learner. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Running, Role: LEADER
I20260809 15:50:21.224356 9788 consensus_queue.cc:260] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:21.225276 9788 raft_consensus.cc:3037] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:21.229028 9701 raft_consensus.cc:2454] T b7526b699d7d4b9680c8f2e98b9c72e2 P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a0709a93f76476a8acbfa4e9eb955d7 in term 2.
I20260809 15:50:21.229995 9788 raft_consensus.cc:2454] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a0709a93f76476a8acbfa4e9eb955d7 in term 2.
I20260809 15:50:21.230180 9830 leader_election.cc:302] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, 4a0709a93f76476a8acbfa4e9eb955d7; no voters:
I20260809 15:50:21.231081 9998 raft_consensus.cc:2781] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:50:21.231542 9998 raft_consensus.cc:686] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 LEADER]: Becoming Leader. State: Replica: 4a0709a93f76476a8acbfa4e9eb955d7, State: Running, Role: LEADER
I20260809 15:50:21.232363 9998 consensus_queue.cc:237] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:21.239565 9584 catalog_manager.cc:5495] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 reported cstate change: term changed from 1 to 2, leader changed from e54dcbceb00649eb944818025b5a969f (127.9.84.2) to 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3). New cstate: current_term: 2 leader_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: HEALTHY } } }
I20260809 15:50:21.285457 9933 consensus_queue.cc:1035] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:21.297260 9929 consensus_queue.cc:1035] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:21.317206 9701 raft_consensus.cc:1261] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Refusing update from remote peer e54dcbceb00649eb944818025b5a969f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:21.318360 9972 consensus_queue.cc:1035] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:21.325434 9871 raft_consensus.cc:3037] T d9963b7dc24a41838578644fa0572566 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:21.332679 9871 raft_consensus.cc:1261] T d9963b7dc24a41838578644fa0572566 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Refusing update from remote peer e54dcbceb00649eb944818025b5a969f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:21.333935 9968 consensus_queue.cc:1035] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:21.465992 9871 raft_consensus.cc:1261] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Refusing update from remote peer e54dcbceb00649eb944818025b5a969f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:21.467646 9968 consensus_queue.cc:1035] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:21.481019 9701 raft_consensus.cc:1261] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Refusing update from remote peer e54dcbceb00649eb944818025b5a969f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:21.483291 9968 consensus_queue.cc:1035] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:50:21.563791 9872 raft_consensus.cc:3037] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:21.568315 9872 raft_consensus.cc:1261] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Refusing update from remote peer 042b068763ff48718439228aebb5cfcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:21.569401 9962 consensus_queue.cc:1035] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:21.578429 9788 raft_consensus.cc:1261] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Refusing update from remote peer 042b068763ff48718439228aebb5cfcb: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:21.580565 9962 consensus_queue.cc:1035] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:50:21.627614 9701 raft_consensus.cc:1261] T b7526b699d7d4b9680c8f2e98b9c72e2 P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Refusing update from remote peer 4a0709a93f76476a8acbfa4e9eb955d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:21.630240 9703 raft_consensus.cc:1261] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Refusing update from remote peer e54dcbceb00649eb944818025b5a969f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:21.630148 9998 consensus_queue.cc:1035] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:50:21.632351 9968 consensus_queue.cc:1035] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:21.645372 9788 raft_consensus.cc:1261] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Refusing update from remote peer 4a0709a93f76476a8acbfa4e9eb955d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:21.647630 9998 consensus_queue.cc:1035] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260809 15:50:21.660665 9872 raft_consensus.cc:1261] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Refusing update from remote peer e54dcbceb00649eb944818025b5a969f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:21.663529 9968 consensus_queue.cc:1035] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:50:23.866353 9552 auto_leader_rebalancer-test.cc:269] The leader distribution is
I20260809 15:50:23.867175 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:23.888129 9701 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3e74d7900d9d485092a9e8b144fca6b0"
dest_uuid: "042b068763ff48718439228aebb5cfcb"
mode: GRACEFUL
new_leader_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
from {username='slave'} at 127.0.0.1:40556
I20260809 15:50:23.888569 9701 raft_consensus.cc:595] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 1 LEADER]: Received request to transfer leadership to TS 4a0709a93f76476a8acbfa4e9eb955d7
I20260809 15:50:23.899663 9788 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a9d11b0e8c644e089dd34200345495c7"
dest_uuid: "e54dcbceb00649eb944818025b5a969f"
mode: GRACEFUL
new_leader_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
from {username='slave'} at 127.0.0.1:43764
I20260809 15:50:23.900156 9788 raft_consensus.cc:595] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Received request to transfer leadership to TS 4a0709a93f76476a8acbfa4e9eb955d7
I20260809 15:50:23.901114 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260809 15:50:23.901541 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:24.220234 9968 raft_consensus.cc:979] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f: : Instructing follower 4a0709a93f76476a8acbfa4e9eb955d7 to start an election
I20260809 15:50:24.220659 10021 raft_consensus.cc:1067] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Signalling peer 4a0709a93f76476a8acbfa4e9eb955d7 to start an election
I20260809 15:50:24.222052 9872 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a9d11b0e8c644e089dd34200345495c7"
dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
from {username='slave'} at 127.0.0.1:41158
I20260809 15:50:24.222538 9872 raft_consensus.cc:482] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:24.222884 9872 raft_consensus.cc:3037] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:24.228312 9872 raft_consensus.cc:504] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:24.229691 9872 leader_election.cc:288] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 3 election: Requested vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899)
I20260809 15:50:24.230581 9701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9d11b0e8c644e089dd34200345495c7" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "042b068763ff48718439228aebb5cfcb"
I20260809 15:50:24.230800 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9d11b0e8c644e089dd34200345495c7" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e54dcbceb00649eb944818025b5a969f"
I20260809 15:50:24.231192 9701 raft_consensus.cc:3037] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:24.231318 9788 raft_consensus.cc:3032] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:50:24.231669 9788 raft_consensus.cc:729] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Running, Role: LEADER
I20260809 15:50:24.232375 9788 consensus_queue.cc:260] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:24.233556 9788 raft_consensus.cc:3037] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:24.237388 9701 raft_consensus.cc:2454] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a0709a93f76476a8acbfa4e9eb955d7 in term 3.
I20260809 15:50:24.238481 9830 leader_election.cc:302] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, 4a0709a93f76476a8acbfa4e9eb955d7; no voters:
I20260809 15:50:24.239148 9999 raft_consensus.cc:2781] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:50:24.239424 9788 raft_consensus.cc:2454] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a0709a93f76476a8acbfa4e9eb955d7 in term 3.
I20260809 15:50:24.240479 9999 raft_consensus.cc:686] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 LEADER]: Becoming Leader. State: Replica: 4a0709a93f76476a8acbfa4e9eb955d7, State: Running, Role: LEADER
I20260809 15:50:24.241300 9999 consensus_queue.cc:237] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:24.247754 9585 catalog_manager.cc:5495] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 reported cstate change: term changed from 2 to 3, leader changed from e54dcbceb00649eb944818025b5a969f (127.9.84.2) to 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3). New cstate: current_term: 3 leader_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: HEALTHY } } }
I20260809 15:50:24.338060 10016 raft_consensus.cc:979] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb: : Instructing follower 4a0709a93f76476a8acbfa4e9eb955d7 to start an election
I20260809 15:50:24.338511 10030 raft_consensus.cc:1067] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 1 LEADER]: Signalling peer 4a0709a93f76476a8acbfa4e9eb955d7 to start an election
I20260809 15:50:24.339917 9872 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3e74d7900d9d485092a9e8b144fca6b0"
dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
from {username='slave'} at 127.0.0.1:41168
I20260809 15:50:24.340373 9872 raft_consensus.cc:482] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:24.340749 9872 raft_consensus.cc:3037] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:24.345266 9872 raft_consensus.cc:504] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } }
I20260809 15:50:24.347435 9872 leader_election.cc:288] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 2 election: Requested vote from peers e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899), 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125)
I20260809 15:50:24.347811 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e74d7900d9d485092a9e8b144fca6b0" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "e54dcbceb00649eb944818025b5a969f"
I20260809 15:50:24.348095 9701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e74d7900d9d485092a9e8b144fca6b0" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "042b068763ff48718439228aebb5cfcb"
I20260809 15:50:24.348408 9788 raft_consensus.cc:3037] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:24.348619 9701 raft_consensus.cc:3032] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:50:24.348978 9701 raft_consensus.cc:729] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Running, Role: LEADER
I20260809 15:50:24.349718 9701 consensus_queue.cc:260] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } }
I20260809 15:50:24.350924 9701 raft_consensus.cc:3037] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:24.354754 9788 raft_consensus.cc:2454] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a0709a93f76476a8acbfa4e9eb955d7 in term 2.
I20260809 15:50:24.355786 9832 leader_election.cc:302] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a0709a93f76476a8acbfa4e9eb955d7, e54dcbceb00649eb944818025b5a969f; no voters:
I20260809 15:50:24.356521 9999 raft_consensus.cc:2781] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:50:24.357008 9999 raft_consensus.cc:686] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 LEADER]: Becoming Leader. State: Replica: 4a0709a93f76476a8acbfa4e9eb955d7, State: Running, Role: LEADER
I20260809 15:50:24.356983 9701 raft_consensus.cc:2454] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a0709a93f76476a8acbfa4e9eb955d7 in term 2.
I20260809 15:50:24.358013 9999 consensus_queue.cc:237] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } }
I20260809 15:50:24.364740 9585 catalog_manager.cc:5495] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 reported cstate change: term changed from 1 to 2, leader changed from 042b068763ff48718439228aebb5cfcb (127.9.84.1) to 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3). New cstate: current_term: 2 leader_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:24.671900 9788 raft_consensus.cc:1261] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Refusing update from remote peer 4a0709a93f76476a8acbfa4e9eb955d7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:50:24.673043 9999 consensus_queue.cc:1035] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:50:24.683506 9701 raft_consensus.cc:1261] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb [term 3 FOLLOWER]: Refusing update from remote peer 4a0709a93f76476a8acbfa4e9eb955d7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:50:24.685113 10034 consensus_queue.cc:1035] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:50:24.755172 9701 raft_consensus.cc:1261] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Refusing update from remote peer 4a0709a93f76476a8acbfa4e9eb955d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:24.756248 10034 consensus_queue.cc:1035] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:24.764477 9788 raft_consensus.cc:1261] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Refusing update from remote peer 4a0709a93f76476a8acbfa4e9eb955d7: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:24.765952 9999 consensus_queue.cc:1035] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:24.902402 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:24.907534 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:24.907938 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:25.908669 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:25.913318 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:25.913655 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:26.914429 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:26.919783 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:26.920184 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:27.921293 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:27.926425 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:27.926833 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:28.927639 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:28.932299 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:28.932673 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:29.933459 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:29.937948 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:29.938375 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:30.939210 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:30.944398 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:30.944782 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:31.945487 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:31.950397 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:31.950771 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:32.951526 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:32.956290 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:32.956688 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:33.981534 9552 auto_leader_rebalancer-test.cc:288] The leader distribution is
I20260809 15:50:33.985520 9872 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3e74d7900d9d485092a9e8b144fca6b0"
dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
mode: GRACEFUL
new_leader_uuid: "e54dcbceb00649eb944818025b5a969f"
from {username='slave'} at 127.0.0.1:41180
I20260809 15:50:33.986227 9872 raft_consensus.cc:595] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 LEADER]: Received request to transfer leadership to TS e54dcbceb00649eb944818025b5a969f
I20260809 15:50:33.990731 9701 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "10055574bc224746bde901e394facf6a"
dest_uuid: "042b068763ff48718439228aebb5cfcb"
mode: GRACEFUL
new_leader_uuid: "e54dcbceb00649eb944818025b5a969f"
from {username='slave'} at 127.0.0.1:40548
I20260809 15:50:33.991274 9701 raft_consensus.cc:595] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 1 LEADER]: Received request to transfer leadership to TS e54dcbceb00649eb944818025b5a969f
I20260809 15:50:33.995591 9701 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "726df19d010d4f008eb75311543d0dd6"
dest_uuid: "042b068763ff48718439228aebb5cfcb"
mode: GRACEFUL
new_leader_uuid: "e54dcbceb00649eb944818025b5a969f"
from {username='slave'} at 127.0.0.1:40548
I20260809 15:50:33.996019 9701 raft_consensus.cc:595] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 2 LEADER]: Received request to transfer leadership to TS e54dcbceb00649eb944818025b5a969f
I20260809 15:50:33.999904 9701 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0fa161c687aa4ee390cca51e913fe50f"
dest_uuid: "042b068763ff48718439228aebb5cfcb"
mode: GRACEFUL
new_leader_uuid: "e54dcbceb00649eb944818025b5a969f"
from {username='slave'} at 127.0.0.1:40548
I20260809 15:50:34.000298 9701 raft_consensus.cc:595] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 1 LEADER]: Received request to transfer leadership to TS e54dcbceb00649eb944818025b5a969f
I20260809 15:50:34.004097 9872 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a9d11b0e8c644e089dd34200345495c7"
dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
mode: GRACEFUL
new_leader_uuid: "e54dcbceb00649eb944818025b5a969f"
from {username='slave'} at 127.0.0.1:41180
I20260809 15:50:34.004541 9872 raft_consensus.cc:595] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 LEADER]: Received request to transfer leadership to TS e54dcbceb00649eb944818025b5a969f
I20260809 15:50:34.051903 10054 raft_consensus.cc:979] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb: : Instructing follower e54dcbceb00649eb944818025b5a969f to start an election
I20260809 15:50:34.052312 10054 raft_consensus.cc:1067] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 2 LEADER]: Signalling peer e54dcbceb00649eb944818025b5a969f to start an election
I20260809 15:50:34.053586 9788 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "726df19d010d4f008eb75311543d0dd6"
dest_uuid: "e54dcbceb00649eb944818025b5a969f"
from {username='slave'} at 127.0.0.1:43750
I20260809 15:50:34.054009 9788 raft_consensus.cc:482] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:34.054355 9788 raft_consensus.cc:3037] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:34.059473 9788 raft_consensus.cc:504] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:34.061013 9788 leader_election.cc:288] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 3 election: Requested vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:34.061775 9701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "726df19d010d4f008eb75311543d0dd6" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "042b068763ff48718439228aebb5cfcb"
I20260809 15:50:34.062068 9872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "726df19d010d4f008eb75311543d0dd6" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
I20260809 15:50:34.062639 9701 raft_consensus.cc:3032] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:50:34.062716 9872 raft_consensus.cc:3037] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:34.062925 9701 raft_consensus.cc:729] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Running, Role: LEADER
I20260809 15:50:34.063633 9701 consensus_queue.cc:260] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:34.064544 9701 raft_consensus.cc:3037] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:34.067380 9872 raft_consensus.cc:2454] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 3.
I20260809 15:50:34.068528 9749 leader_election.cc:302] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a0709a93f76476a8acbfa4e9eb955d7, e54dcbceb00649eb944818025b5a969f; no voters:
I20260809 15:50:34.068778 9701 raft_consensus.cc:2454] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 3.
I20260809 15:50:34.069376 10051 raft_consensus.cc:2781] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:50:34.070604 10051 raft_consensus.cc:686] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 3 LEADER]: Becoming Leader. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Running, Role: LEADER
I20260809 15:50:34.071933 10055 raft_consensus.cc:979] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb: : Instructing follower e54dcbceb00649eb944818025b5a969f to start an election
I20260809 15:50:34.071493 10051 consensus_queue.cc:237] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:34.072386 10055 raft_consensus.cc:1067] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 1 LEADER]: Signalling peer e54dcbceb00649eb944818025b5a969f to start an election
I20260809 15:50:34.073781 9788 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0fa161c687aa4ee390cca51e913fe50f"
dest_uuid: "e54dcbceb00649eb944818025b5a969f"
from {username='slave'} at 127.0.0.1:43750
I20260809 15:50:34.074327 9788 raft_consensus.cc:482] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:34.074725 9788 raft_consensus.cc:3037] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:34.078802 9788 raft_consensus.cc:504] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:34.080528 9788 leader_election.cc:288] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 2 election: Requested vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:34.079732 9585 catalog_manager.cc:5495] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f reported cstate change: term changed from 2 to 3, leader changed from 042b068763ff48718439228aebb5cfcb (127.9.84.1) to e54dcbceb00649eb944818025b5a969f (127.9.84.2). New cstate: current_term: 3 leader_uuid: "e54dcbceb00649eb944818025b5a969f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:34.081053 9701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fa161c687aa4ee390cca51e913fe50f" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "042b068763ff48718439228aebb5cfcb"
I20260809 15:50:34.081547 9701 raft_consensus.cc:3032] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:50:34.081881 9701 raft_consensus.cc:729] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Running, Role: LEADER
I20260809 15:50:34.081883 9872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fa161c687aa4ee390cca51e913fe50f" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
I20260809 15:50:34.082615 9872 raft_consensus.cc:3037] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:34.082748 9701 consensus_queue.cc:260] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:34.084142 9701 raft_consensus.cc:3037] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:34.089128 9701 raft_consensus.cc:2454] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 2.
I20260809 15:50:34.089493 9872 raft_consensus.cc:2454] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 2.
I20260809 15:50:34.090361 9747 leader_election.cc:302] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, e54dcbceb00649eb944818025b5a969f; no voters:
I20260809 15:50:34.091634 10051 raft_consensus.cc:2781] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:50:34.091902 10061 raft_consensus.cc:979] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7: : Instructing follower e54dcbceb00649eb944818025b5a969f to start an election
I20260809 15:50:34.092487 10051 raft_consensus.cc:686] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Becoming Leader. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Running, Role: LEADER
I20260809 15:50:34.092551 10046 raft_consensus.cc:1067] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 LEADER]: Signalling peer e54dcbceb00649eb944818025b5a969f to start an election
I20260809 15:50:34.093271 10051 consensus_queue.cc:237] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:34.094094 9788 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a9d11b0e8c644e089dd34200345495c7"
dest_uuid: "e54dcbceb00649eb944818025b5a969f"
from {username='slave'} at 127.0.0.1:43748
I20260809 15:50:34.094594 9788 raft_consensus.cc:482] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:34.094930 9788 raft_consensus.cc:3037] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:50:34.101056 9788 raft_consensus.cc:504] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:34.102874 9788 leader_election.cc:288] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 4 election: Requested vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:34.103166 9701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9d11b0e8c644e089dd34200345495c7" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "042b068763ff48718439228aebb5cfcb"
I20260809 15:50:34.103443 9872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a9d11b0e8c644e089dd34200345495c7" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
I20260809 15:50:34.103688 9701 raft_consensus.cc:3037] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:50:34.103976 9872 raft_consensus.cc:3032] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:50:34.104305 9872 raft_consensus.cc:729] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4a0709a93f76476a8acbfa4e9eb955d7, State: Running, Role: LEADER
I20260809 15:50:34.105049 9872 consensus_queue.cc:260] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:34.105533 9585 catalog_manager.cc:5495] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f reported cstate change: term changed from 1 to 2, leader changed from 042b068763ff48718439228aebb5cfcb (127.9.84.1) to e54dcbceb00649eb944818025b5a969f (127.9.84.2). New cstate: current_term: 2 leader_uuid: "e54dcbceb00649eb944818025b5a969f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:34.106295 9872 raft_consensus.cc:3037] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:50:34.109342 9701 raft_consensus.cc:2454] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 4.
I20260809 15:50:34.110244 9747 leader_election.cc:302] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, e54dcbceb00649eb944818025b5a969f; no voters:
I20260809 15:50:34.111001 10051 raft_consensus.cc:2781] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:50:34.111438 10051 raft_consensus.cc:686] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 4 LEADER]: Becoming Leader. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Running, Role: LEADER
I20260809 15:50:34.112162 9872 raft_consensus.cc:2454] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 4.
I20260809 15:50:34.112232 10051 consensus_queue.cc:237] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:34.118760 9585 catalog_manager.cc:5495] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f reported cstate change: term changed from 3 to 4, leader changed from 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3) to e54dcbceb00649eb944818025b5a969f (127.9.84.2). New cstate: current_term: 4 leader_uuid: "e54dcbceb00649eb944818025b5a969f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:34.318758 10061 raft_consensus.cc:979] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7: : Instructing follower e54dcbceb00649eb944818025b5a969f to start an election
I20260809 15:50:34.319253 10046 raft_consensus.cc:1067] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 LEADER]: Signalling peer e54dcbceb00649eb944818025b5a969f to start an election
I20260809 15:50:34.320700 9788 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3e74d7900d9d485092a9e8b144fca6b0"
dest_uuid: "e54dcbceb00649eb944818025b5a969f"
from {username='slave'} at 127.0.0.1:43748
I20260809 15:50:34.321197 9788 raft_consensus.cc:482] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:34.321465 9788 raft_consensus.cc:3037] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:34.325327 9788 raft_consensus.cc:504] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } }
I20260809 15:50:34.326740 9788 leader_election.cc:288] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 3 election: Requested vote from peers 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553), 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125)
I20260809 15:50:34.327553 9872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e74d7900d9d485092a9e8b144fca6b0" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
I20260809 15:50:34.327741 9701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e74d7900d9d485092a9e8b144fca6b0" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "042b068763ff48718439228aebb5cfcb"
I20260809 15:50:34.328058 9872 raft_consensus.cc:3032] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:50:34.328228 9701 raft_consensus.cc:3037] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:34.328383 9872 raft_consensus.cc:729] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4a0709a93f76476a8acbfa4e9eb955d7, State: Running, Role: LEADER
I20260809 15:50:34.329068 9872 consensus_queue.cc:260] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } }
I20260809 15:50:34.329941 9872 raft_consensus.cc:3037] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:34.332182 9701 raft_consensus.cc:2454] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 3.
I20260809 15:50:34.333004 9747 leader_election.cc:302] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, e54dcbceb00649eb944818025b5a969f; no voters:
I20260809 15:50:34.333716 10051 raft_consensus.cc:2781] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:50:34.334124 10051 raft_consensus.cc:686] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 3 LEADER]: Becoming Leader. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Running, Role: LEADER
I20260809 15:50:34.334758 9872 raft_consensus.cc:2454] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 3.
I20260809 15:50:34.334903 10051 consensus_queue.cc:237] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } }
I20260809 15:50:34.341518 10054 raft_consensus.cc:979] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb: : Instructing follower e54dcbceb00649eb944818025b5a969f to start an election
I20260809 15:50:34.341972 10055 raft_consensus.cc:1067] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 1 LEADER]: Signalling peer e54dcbceb00649eb944818025b5a969f to start an election
I20260809 15:50:34.342054 9585 catalog_manager.cc:5495] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f reported cstate change: term changed from 2 to 3, leader changed from 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3) to e54dcbceb00649eb944818025b5a969f (127.9.84.2). New cstate: current_term: 3 leader_uuid: "e54dcbceb00649eb944818025b5a969f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:34.343533 9788 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "10055574bc224746bde901e394facf6a"
dest_uuid: "e54dcbceb00649eb944818025b5a969f"
from {username='slave'} at 127.0.0.1:43750
I20260809 15:50:34.344017 9788 raft_consensus.cc:482] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:34.344339 9788 raft_consensus.cc:3037] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:34.349722 9788 raft_consensus.cc:504] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:34.351529 9788 leader_election.cc:288] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 2 election: Requested vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:34.352150 9701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10055574bc224746bde901e394facf6a" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "042b068763ff48718439228aebb5cfcb"
I20260809 15:50:34.352658 9701 raft_consensus.cc:3032] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:50:34.352500 9872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10055574bc224746bde901e394facf6a" candidate_uuid: "e54dcbceb00649eb944818025b5a969f" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
I20260809 15:50:34.352891 9701 raft_consensus.cc:729] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Running, Role: LEADER
I20260809 15:50:34.353220 9872 raft_consensus.cc:3037] T 10055574bc224746bde901e394facf6a P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:34.353483 9701 consensus_queue.cc:260] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:34.354503 9701 raft_consensus.cc:3037] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:34.357609 9872 raft_consensus.cc:2454] T 10055574bc224746bde901e394facf6a P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 2.
I20260809 15:50:34.358449 9749 leader_election.cc:302] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a0709a93f76476a8acbfa4e9eb955d7, e54dcbceb00649eb944818025b5a969f; no voters:
I20260809 15:50:34.358649 9701 raft_consensus.cc:2454] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e54dcbceb00649eb944818025b5a969f in term 2.
I20260809 15:50:34.359181 10051 raft_consensus.cc:2781] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:50:34.359555 10051 raft_consensus.cc:686] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Becoming Leader. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Running, Role: LEADER
I20260809 15:50:34.360241 10051 consensus_queue.cc:237] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:34.366698 9585 catalog_manager.cc:5495] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f reported cstate change: term changed from 1 to 2, leader changed from 042b068763ff48718439228aebb5cfcb (127.9.84.1) to e54dcbceb00649eb944818025b5a969f (127.9.84.2). New cstate: current_term: 2 leader_uuid: "e54dcbceb00649eb944818025b5a969f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:34.495729 9701 raft_consensus.cc:1261] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Refusing update from remote peer e54dcbceb00649eb944818025b5a969f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:34.496416 9703 raft_consensus.cc:1261] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 3 FOLLOWER]: Refusing update from remote peer e54dcbceb00649eb944818025b5a969f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:50:34.496832 10051 consensus_queue.cc:1035] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:34.497771 10056 consensus_queue.cc:1035] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:50:34.507594 9872 raft_consensus.cc:1261] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Refusing update from remote peer e54dcbceb00649eb944818025b5a969f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:34.511870 10072 consensus_queue.cc:1035] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:50:34.517205 9872 raft_consensus.cc:1261] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Refusing update from remote peer e54dcbceb00649eb944818025b5a969f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:50:34.519125 10051 consensus_queue.cc:1035] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:50:34.598932 9701 raft_consensus.cc:1261] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb [term 4 FOLLOWER]: Refusing update from remote peer e54dcbceb00649eb944818025b5a969f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:50:34.600193 10073 consensus_queue.cc:1035] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:50:34.610422 9872 raft_consensus.cc:1261] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 4 FOLLOWER]: Refusing update from remote peer e54dcbceb00649eb944818025b5a969f: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:50:34.612497 10073 consensus_queue.cc:1035] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:50:34.794173 9872 raft_consensus.cc:1261] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Refusing update from remote peer e54dcbceb00649eb944818025b5a969f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:50:34.795437 10073 consensus_queue.cc:1035] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:50:34.803665 9701 raft_consensus.cc:1261] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 3 FOLLOWER]: Refusing update from remote peer e54dcbceb00649eb944818025b5a969f: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:50:34.805094 10056 consensus_queue.cc:1035] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:50:34.873289 9872 raft_consensus.cc:1261] T 10055574bc224746bde901e394facf6a P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Refusing update from remote peer e54dcbceb00649eb944818025b5a969f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:34.874612 10056 consensus_queue.cc:1035] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:34.886837 9701 raft_consensus.cc:1261] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Refusing update from remote peer e54dcbceb00649eb944818025b5a969f: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:34.888038 10073 consensus_queue.cc:1035] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [LEADER]: Connected to new peer: Peer: permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:37.038142 9552 auto_leader_rebalancer-test.cc:303] The leader distribution is
I20260809 15:50:37.038697 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:37.044855 9788 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0fa161c687aa4ee390cca51e913fe50f"
dest_uuid: "e54dcbceb00649eb944818025b5a969f"
mode: GRACEFUL
new_leader_uuid: "042b068763ff48718439228aebb5cfcb"
from {username='slave'} at 127.0.0.1:43764
I20260809 15:50:37.045320 9788 raft_consensus.cc:595] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Received request to transfer leadership to TS 042b068763ff48718439228aebb5cfcb
I20260809 15:50:37.047508 9788 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3e74d7900d9d485092a9e8b144fca6b0"
dest_uuid: "e54dcbceb00649eb944818025b5a969f"
mode: GRACEFUL
new_leader_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
from {username='slave'} at 127.0.0.1:43764
I20260809 15:50:37.047911 9788 raft_consensus.cc:595] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 3 LEADER]: Received request to transfer leadership to TS 4a0709a93f76476a8acbfa4e9eb955d7
I20260809 15:50:37.049706 9788 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "691afd4416c84d7d8e595a5b7c5e7d4b"
dest_uuid: "e54dcbceb00649eb944818025b5a969f"
mode: GRACEFUL
new_leader_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
from {username='slave'} at 127.0.0.1:43764
I20260809 15:50:37.050148 9788 raft_consensus.cc:595] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Received request to transfer leadership to TS 4a0709a93f76476a8acbfa4e9eb955d7
I20260809 15:50:37.051935 9788 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "726df19d010d4f008eb75311543d0dd6"
dest_uuid: "e54dcbceb00649eb944818025b5a969f"
mode: GRACEFUL
new_leader_uuid: "042b068763ff48718439228aebb5cfcb"
from {username='slave'} at 127.0.0.1:43764
I20260809 15:50:37.052330 9788 raft_consensus.cc:595] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 3 LEADER]: Received request to transfer leadership to TS 042b068763ff48718439228aebb5cfcb
I20260809 15:50:37.054128 9788 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "d9963b7dc24a41838578644fa0572566"
dest_uuid: "e54dcbceb00649eb944818025b5a969f"
mode: GRACEFUL
new_leader_uuid: "042b068763ff48718439228aebb5cfcb"
from {username='slave'} at 127.0.0.1:43764
I20260809 15:50:37.054507 9788 raft_consensus.cc:595] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Received request to transfer leadership to TS 042b068763ff48718439228aebb5cfcb
I20260809 15:50:37.055450 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260809 15:50:37.055836 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:37.265056 10085 raft_consensus.cc:979] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f: : Instructing follower 4a0709a93f76476a8acbfa4e9eb955d7 to start an election
I20260809 15:50:37.265496 10084 raft_consensus.cc:1067] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 3 LEADER]: Signalling peer 4a0709a93f76476a8acbfa4e9eb955d7 to start an election
I20260809 15:50:37.266816 9872 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3e74d7900d9d485092a9e8b144fca6b0"
dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
from {username='slave'} at 127.0.0.1:41158
I20260809 15:50:37.267263 9872 raft_consensus.cc:482] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:37.267522 9872 raft_consensus.cc:3037] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:50:37.271427 9872 raft_consensus.cc:504] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } }
I20260809 15:50:37.273214 9872 leader_election.cc:288] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 4 election: Requested vote from peers e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899), 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125)
I20260809 15:50:37.273947 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e74d7900d9d485092a9e8b144fca6b0" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e54dcbceb00649eb944818025b5a969f"
I20260809 15:50:37.274233 9701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e74d7900d9d485092a9e8b144fca6b0" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "042b068763ff48718439228aebb5cfcb"
I20260809 15:50:37.274488 9788 raft_consensus.cc:3032] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:50:37.274737 9701 raft_consensus.cc:3037] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:50:37.274820 9788 raft_consensus.cc:729] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Running, Role: LEADER
I20260809 15:50:37.275499 9788 consensus_queue.cc:260] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } }
I20260809 15:50:37.276311 9788 raft_consensus.cc:3037] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:50:37.278533 9701 raft_consensus.cc:2454] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a0709a93f76476a8acbfa4e9eb955d7 in term 4.
I20260809 15:50:37.279639 9830 leader_election.cc:302] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, 4a0709a93f76476a8acbfa4e9eb955d7; no voters:
I20260809 15:50:37.280191 9788 raft_consensus.cc:2454] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a0709a93f76476a8acbfa4e9eb955d7 in term 4.
I20260809 15:50:37.280287 10046 raft_consensus.cc:2781] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:50:37.281234 10046 raft_consensus.cc:686] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 4 LEADER]: Becoming Leader. State: Replica: 4a0709a93f76476a8acbfa4e9eb955d7, State: Running, Role: LEADER
I20260809 15:50:37.281971 10046 consensus_queue.cc:237] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } }
I20260809 15:50:37.288153 9585 catalog_manager.cc:5495] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 reported cstate change: term changed from 3 to 4, leader changed from e54dcbceb00649eb944818025b5a969f (127.9.84.2) to 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3). New cstate: current_term: 4 leader_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:37.304418 10085 raft_consensus.cc:979] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f: : Instructing follower 042b068763ff48718439228aebb5cfcb to start an election
I20260809 15:50:37.304816 10084 raft_consensus.cc:1067] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Signalling peer 042b068763ff48718439228aebb5cfcb to start an election
I20260809 15:50:37.306221 9701 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0fa161c687aa4ee390cca51e913fe50f"
dest_uuid: "042b068763ff48718439228aebb5cfcb"
from {username='slave'} at 127.0.0.1:40520
I20260809 15:50:37.306576 9701 raft_consensus.cc:482] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:37.306779 9701 raft_consensus.cc:3037] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:37.310412 9701 raft_consensus.cc:504] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:37.311936 9701 leader_election.cc:288] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 3 election: Requested vote from peers e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:37.312691 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fa161c687aa4ee390cca51e913fe50f" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e54dcbceb00649eb944818025b5a969f"
I20260809 15:50:37.312836 9872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0fa161c687aa4ee390cca51e913fe50f" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
I20260809 15:50:37.313261 9872 raft_consensus.cc:3037] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:37.313225 9788 raft_consensus.cc:3032] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:50:37.313725 9788 raft_consensus.cc:729] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Running, Role: LEADER
I20260809 15:50:37.314407 9788 consensus_queue.cc:260] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:37.315449 9788 raft_consensus.cc:3037] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:37.318945 9872 raft_consensus.cc:2454] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 042b068763ff48718439228aebb5cfcb in term 3.
I20260809 15:50:37.319954 9661 leader_election.cc:302] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, 4a0709a93f76476a8acbfa4e9eb955d7; no voters:
I20260809 15:50:37.320605 9788 raft_consensus.cc:2454] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 042b068763ff48718439228aebb5cfcb in term 3.
I20260809 15:50:37.321439 10092 raft_consensus.cc:2781] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:50:37.322503 10092 raft_consensus.cc:686] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 3 LEADER]: Becoming Leader. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Running, Role: LEADER
I20260809 15:50:37.323228 10092 consensus_queue.cc:237] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:37.329560 9585 catalog_manager.cc:5495] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb reported cstate change: term changed from 2 to 3, leader changed from e54dcbceb00649eb944818025b5a969f (127.9.84.2) to 042b068763ff48718439228aebb5cfcb (127.9.84.1). New cstate: current_term: 3 leader_uuid: "042b068763ff48718439228aebb5cfcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:37.447829 10085 raft_consensus.cc:979] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f: : Instructing follower 042b068763ff48718439228aebb5cfcb to start an election
I20260809 15:50:37.448621 10084 raft_consensus.cc:1067] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Signalling peer 042b068763ff48718439228aebb5cfcb to start an election
I20260809 15:50:37.450731 9701 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "d9963b7dc24a41838578644fa0572566"
dest_uuid: "042b068763ff48718439228aebb5cfcb"
from {username='slave'} at 127.0.0.1:40520
I20260809 15:50:37.451457 9701 raft_consensus.cc:482] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:37.451974 9701 raft_consensus.cc:3037] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:37.453673 10085 raft_consensus.cc:979] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f: : Instructing follower 042b068763ff48718439228aebb5cfcb to start an election
I20260809 15:50:37.454340 10084 raft_consensus.cc:1067] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 3 LEADER]: Signalling peer 042b068763ff48718439228aebb5cfcb to start an election
I20260809 15:50:37.456086 9703 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "726df19d010d4f008eb75311543d0dd6"
dest_uuid: "042b068763ff48718439228aebb5cfcb"
from {username='slave'} at 127.0.0.1:40520
I20260809 15:50:37.456333 10085 raft_consensus.cc:979] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f: : Instructing follower 4a0709a93f76476a8acbfa4e9eb955d7 to start an election
I20260809 15:50:37.456704 9703 raft_consensus.cc:482] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:37.456892 10084 raft_consensus.cc:1067] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Signalling peer 4a0709a93f76476a8acbfa4e9eb955d7 to start an election
I20260809 15:50:37.457273 9703 raft_consensus.cc:3037] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:50:37.458493 9872 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "691afd4416c84d7d8e595a5b7c5e7d4b"
dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
from {username='slave'} at 127.0.0.1:41158
I20260809 15:50:37.459004 9872 raft_consensus.cc:482] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:37.459277 9872 raft_consensus.cc:3037] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:37.461038 9701 raft_consensus.cc:504] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:37.463580 9701 leader_election.cc:288] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 3 election: Requested vote from peers e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:37.464784 9703 raft_consensus.cc:504] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:37.465173 9872 raft_consensus.cc:504] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:37.465643 9788 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9963b7dc24a41838578644fa0572566" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e54dcbceb00649eb944818025b5a969f"
I20260809 15:50:37.465807 9871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9963b7dc24a41838578644fa0572566" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
I20260809 15:50:37.466596 9788 raft_consensus.cc:3032] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:50:37.466784 9871 raft_consensus.cc:3037] T d9963b7dc24a41838578644fa0572566 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:37.466951 9788 raft_consensus.cc:729] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Running, Role: LEADER
I20260809 15:50:37.467425 9872 leader_election.cc:288] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 3 election: Requested vote from peers 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125), e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899)
I20260809 15:50:37.467780 9788 consensus_queue.cc:260] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:37.468532 9703 leader_election.cc:288] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 4 election: Requested vote from peers e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899), 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3:39553)
I20260809 15:50:37.469271 9788 raft_consensus.cc:3037] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:37.469448 9701 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691afd4416c84d7d8e595a5b7c5e7d4b" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "042b068763ff48718439228aebb5cfcb"
I20260809 15:50:37.470024 9701 raft_consensus.cc:3037] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:37.471573 9785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "691afd4416c84d7d8e595a5b7c5e7d4b" candidate_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "e54dcbceb00649eb944818025b5a969f"
I20260809 15:50:37.471978 9787 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "726df19d010d4f008eb75311543d0dd6" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "e54dcbceb00649eb944818025b5a969f"
I20260809 15:50:37.472584 9787 raft_consensus.cc:3032] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:50:37.472895 9787 raft_consensus.cc:729] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 3 LEADER]: Becoming Follower/Learner. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Running, Role: LEADER
I20260809 15:50:37.473631 9787 consensus_queue.cc:260] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:37.475240 9787 raft_consensus.cc:3037] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:50:37.475451 9871 raft_consensus.cc:2454] T d9963b7dc24a41838578644fa0572566 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 042b068763ff48718439228aebb5cfcb in term 3.
I20260809 15:50:37.476256 9701 raft_consensus.cc:2454] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a0709a93f76476a8acbfa4e9eb955d7 in term 3.
I20260809 15:50:37.476796 9872 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "726df19d010d4f008eb75311543d0dd6" candidate_uuid: "042b068763ff48718439228aebb5cfcb" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4a0709a93f76476a8acbfa4e9eb955d7"
I20260809 15:50:37.477228 9788 raft_consensus.cc:2454] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 042b068763ff48718439228aebb5cfcb in term 3.
I20260809 15:50:37.477417 9872 raft_consensus.cc:3037] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:50:37.472170 9785 raft_consensus.cc:3032] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:50:37.478511 9830 leader_election.cc:302] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, 4a0709a93f76476a8acbfa4e9eb955d7; no voters:
I20260809 15:50:37.478878 9785 raft_consensus.cc:729] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Becoming Follower/Learner. State: Replica: e54dcbceb00649eb944818025b5a969f, State: Running, Role: LEADER
I20260809 15:50:37.478529 9661 leader_election.cc:302] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, e54dcbceb00649eb944818025b5a969f; no voters:
I20260809 15:50:37.479817 10046 raft_consensus.cc:2781] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:50:37.480356 10046 raft_consensus.cc:686] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 LEADER]: Becoming Leader. State: Replica: 4a0709a93f76476a8acbfa4e9eb955d7, State: Running, Role: LEADER
I20260809 15:50:37.479964 9785 consensus_queue.cc:260] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:37.481338 9785 raft_consensus.cc:3037] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:37.481482 10046 consensus_queue.cc:237] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:37.482254 9787 raft_consensus.cc:2454] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 042b068763ff48718439228aebb5cfcb in term 4.
I20260809 15:50:37.482468 10092 raft_consensus.cc:2781] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:50:37.483232 10092 raft_consensus.cc:686] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 3 LEADER]: Becoming Leader. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Running, Role: LEADER
I20260809 15:50:37.483285 9661 leader_election.cc:302] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 042b068763ff48718439228aebb5cfcb, e54dcbceb00649eb944818025b5a969f; no voters:
I20260809 15:50:37.484321 10092 consensus_queue.cc:237] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:37.485131 10093 raft_consensus.cc:2781] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:50:37.488067 9785 raft_consensus.cc:2454] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a0709a93f76476a8acbfa4e9eb955d7 in term 3.
I20260809 15:50:37.494222 9585 catalog_manager.cc:5495] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb reported cstate change: term changed from 2 to 3, leader changed from e54dcbceb00649eb944818025b5a969f (127.9.84.2) to 042b068763ff48718439228aebb5cfcb (127.9.84.1). New cstate: current_term: 3 leader_uuid: "042b068763ff48718439228aebb5cfcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:37.498322 10093 raft_consensus.cc:686] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 4 LEADER]: Becoming Leader. State: Replica: 042b068763ff48718439228aebb5cfcb, State: Running, Role: LEADER
I20260809 15:50:37.499019 9872 raft_consensus.cc:2454] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 042b068763ff48718439228aebb5cfcb in term 4.
I20260809 15:50:37.499460 10093 consensus_queue.cc:237] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } }
I20260809 15:50:37.499383 9587 catalog_manager.cc:5495] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 reported cstate change: term changed from 2 to 3, leader changed from e54dcbceb00649eb944818025b5a969f (127.9.84.2) to 4a0709a93f76476a8acbfa4e9eb955d7 (127.9.84.3). New cstate: current_term: 3 leader_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: HEALTHY } } }
I20260809 15:50:37.512866 9587 catalog_manager.cc:5495] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb reported cstate change: term changed from 3 to 4, leader changed from e54dcbceb00649eb944818025b5a969f (127.9.84.2) to 042b068763ff48718439228aebb5cfcb (127.9.84.1). New cstate: current_term: 4 leader_uuid: "042b068763ff48718439228aebb5cfcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:37.736507 9785 raft_consensus.cc:1261] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Refusing update from remote peer 042b068763ff48718439228aebb5cfcb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:50:37.737779 10093 consensus_queue.cc:1035] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:50:37.750816 9872 raft_consensus.cc:1261] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Refusing update from remote peer 042b068763ff48718439228aebb5cfcb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:50:37.751960 10093 consensus_queue.cc:1035] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:50:37.795333 9701 raft_consensus.cc:1261] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 4 FOLLOWER]: Refusing update from remote peer 4a0709a93f76476a8acbfa4e9eb955d7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:50:37.796319 10046 consensus_queue.cc:1035] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:50:37.806532 9785 raft_consensus.cc:1261] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 4 FOLLOWER]: Refusing update from remote peer 4a0709a93f76476a8acbfa4e9eb955d7: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:50:37.808149 10086 consensus_queue.cc:1035] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:50:37.862012 9701 raft_consensus.cc:1261] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [term 3 FOLLOWER]: Refusing update from remote peer 4a0709a93f76476a8acbfa4e9eb955d7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:50:37.863123 10086 consensus_queue.cc:1035] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "042b068763ff48718439228aebb5cfcb" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 35125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:50:37.871394 9785 raft_consensus.cc:1261] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Refusing update from remote peer 4a0709a93f76476a8acbfa4e9eb955d7: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:50:37.873536 10111 consensus_queue.cc:1035] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:50:38.038663 9785 raft_consensus.cc:1261] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Refusing update from remote peer 042b068763ff48718439228aebb5cfcb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:50:38.039707 10093 consensus_queue.cc:1035] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:50:38.048169 9872 raft_consensus.cc:1261] T d9963b7dc24a41838578644fa0572566 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Refusing update from remote peer 042b068763ff48718439228aebb5cfcb: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:50:38.049568 10093 consensus_queue.cc:1035] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:50:38.050058 9872 raft_consensus.cc:1261] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 4 FOLLOWER]: Refusing update from remote peer 042b068763ff48718439228aebb5cfcb: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:50:38.051270 10093 consensus_queue.cc:1035] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a0709a93f76476a8acbfa4e9eb955d7" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39553 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:50:38.057082 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:38.066417 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:38.066956 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:38.069954 9785 raft_consensus.cc:1261] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 4 FOLLOWER]: Refusing update from remote peer 042b068763ff48718439228aebb5cfcb: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:50:38.071683 10096 consensus_queue.cc:1035] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e54dcbceb00649eb944818025b5a969f" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 42899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:50:39.067935 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:39.073132 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:39.073446 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:40.074226 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:40.078675 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:40.078999 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:41.079816 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:41.084295 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:41.084615 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:42.085410 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:42.089831 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:42.090328 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:43.091030 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:43.095947 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:43.096328 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:44.097083 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:44.101650 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:44.102039 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:45.102854 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:45.109213 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:45.109670 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:46.110448 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:46.115475 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:46.115859 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:47.144796 9552 auto_leader_rebalancer-test.cc:314] The leader distribution is
I20260809 15:50:47.145588 9552 tablet_server.cc:178] TabletServer@127.9.84.1:0 shutting down...
W20260809 15:50:47.151731 9747 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer
W20260809 15:50:47.156854 9747 consensus_peers.cc:487] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f -> Peer 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125): Couldn't send request to peer 042b068763ff48718439228aebb5cfcb. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:50:47.186445 9552 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260809 15:50:47.187191 9552 tablet_replica.cc:330] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb: stopping tablet replica
I20260809 15:50:47.188246 9552 raft_consensus.cc:2229] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 15:50:47.188952 9552 raft_consensus.cc:2258] T 3e74d7900d9d485092a9e8b144fca6b0 P 042b068763ff48718439228aebb5cfcb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.191159 9552 tablet_replica.cc:330] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb: stopping tablet replica
I20260809 15:50:47.191604 9552 raft_consensus.cc:2229] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 4 LEADER]: Raft consensus shutting down.
I20260809 15:50:47.192252 9552 raft_consensus.cc:2258] T 726df19d010d4f008eb75311543d0dd6 P 042b068763ff48718439228aebb5cfcb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.193636 9552 tablet_replica.cc:330] T 90c5ba0a4aea4aebaeecf735050a4528 P 042b068763ff48718439228aebb5cfcb: stopping tablet replica
I20260809 15:50:47.194176 9552 raft_consensus.cc:2229] T 90c5ba0a4aea4aebaeecf735050a4528 P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:50:47.194504 9552 raft_consensus.cc:2258] T 90c5ba0a4aea4aebaeecf735050a4528 P 042b068763ff48718439228aebb5cfcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.196099 9552 tablet_replica.cc:330] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb: stopping tablet replica
I20260809 15:50:47.196557 9552 raft_consensus.cc:2229] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 15:50:47.196889 9552 raft_consensus.cc:2258] T 10055574bc224746bde901e394facf6a P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.198483 9552 tablet_replica.cc:330] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb: stopping tablet replica
I20260809 15:50:47.198896 9552 raft_consensus.cc:2229] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 3 LEADER]: Raft consensus shutting down.
I20260809 15:50:47.199576 9552 raft_consensus.cc:2258] T 0fa161c687aa4ee390cca51e913fe50f P 042b068763ff48718439228aebb5cfcb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.201143 9552 tablet_replica.cc:330] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb: stopping tablet replica
I20260809 15:50:47.201572 9552 raft_consensus.cc:2229] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 3 LEADER]: Raft consensus shutting down.
I20260809 15:50:47.202270 9552 raft_consensus.cc:2258] T d9963b7dc24a41838578644fa0572566 P 042b068763ff48718439228aebb5cfcb [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.203697 9552 tablet_replica.cc:330] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb: stopping tablet replica
I20260809 15:50:47.204092 9552 raft_consensus.cc:2229] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 15:50:47.204412 9552 raft_consensus.cc:2258] T a9d11b0e8c644e089dd34200345495c7 P 042b068763ff48718439228aebb5cfcb [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.205672 9552 tablet_replica.cc:330] T b7526b699d7d4b9680c8f2e98b9c72e2 P 042b068763ff48718439228aebb5cfcb: stopping tablet replica
I20260809 15:50:47.206053 9552 raft_consensus.cc:2229] T b7526b699d7d4b9680c8f2e98b9c72e2 P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 15:50:47.206436 9552 raft_consensus.cc:2258] T b7526b699d7d4b9680c8f2e98b9c72e2 P 042b068763ff48718439228aebb5cfcb [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.207782 9552 tablet_replica.cc:330] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb: stopping tablet replica
I20260809 15:50:47.208192 9552 raft_consensus.cc:2229] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 15:50:47.208487 9552 raft_consensus.cc:2258] T 691afd4416c84d7d8e595a5b7c5e7d4b P 042b068763ff48718439228aebb5cfcb [term 3 FOLLOWER]: Raft consensus is shut down!
W20260809 15:50:47.283187 9747 consensus_peers.cc:487] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f -> Peer 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125): Couldn't send request to peer 042b068763ff48718439228aebb5cfcb. Status: Network error: Client connection negotiation failed: client connection to 127.9.84.1:35125: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
W20260809 15:50:47.292547 9747 consensus_peers.cc:487] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f -> Peer 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125): Couldn't send request to peer 042b068763ff48718439228aebb5cfcb. Status: Network error: Client connection negotiation failed: client connection to 127.9.84.1:35125: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 15:50:47.296367 9552 tablet_server.cc:194] TabletServer@127.9.84.1:0 shutdown complete.
I20260809 15:50:47.312872 9552 tablet_server.cc:178] TabletServer@127.9.84.2:0 shutting down...
I20260809 15:50:47.340454 9552 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260809 15:50:47.341118 9552 tablet_replica.cc:330] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f: stopping tablet replica
I20260809 15:50:47.341692 9552 raft_consensus.cc:2229] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 15:50:47.342206 9552 raft_consensus.cc:2258] T 726df19d010d4f008eb75311543d0dd6 P e54dcbceb00649eb944818025b5a969f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.343995 9552 tablet_replica.cc:330] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f: stopping tablet replica
I20260809 15:50:47.344504 9552 raft_consensus.cc:2229] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:50:47.345259 9552 raft_consensus.cc:2258] T 90c5ba0a4aea4aebaeecf735050a4528 P e54dcbceb00649eb944818025b5a969f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.346920 9552 tablet_replica.cc:330] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f: stopping tablet replica
I20260809 15:50:47.347378 9552 raft_consensus.cc:2229] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 15:50:47.347765 9552 raft_consensus.cc:2258] T 691afd4416c84d7d8e595a5b7c5e7d4b P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.349699 9552 tablet_replica.cc:330] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f: stopping tablet replica
I20260809 15:50:47.350343 9552 raft_consensus.cc:2229] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 15:50:47.350833 9552 raft_consensus.cc:2258] T b7526b699d7d4b9680c8f2e98b9c72e2 P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.353142 9552 tablet_replica.cc:330] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f: stopping tablet replica
I20260809 15:50:47.353677 9552 raft_consensus.cc:2229] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 4 LEADER]: Raft consensus shutting down.
I20260809 15:50:47.354424 9552 raft_consensus.cc:2258] T a9d11b0e8c644e089dd34200345495c7 P e54dcbceb00649eb944818025b5a969f [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.356160 9552 tablet_replica.cc:330] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f: stopping tablet replica
I20260809 15:50:47.356645 9552 raft_consensus.cc:2229] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 15:50:47.357090 9552 raft_consensus.cc:2258] T d9963b7dc24a41838578644fa0572566 P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.358464 9552 tablet_replica.cc:330] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f: stopping tablet replica
I20260809 15:50:47.358899 9552 raft_consensus.cc:2229] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 15:50:47.359282 9552 raft_consensus.cc:2258] T 0fa161c687aa4ee390cca51e913fe50f P e54dcbceb00649eb944818025b5a969f [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.360694 9552 tablet_replica.cc:330] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f: stopping tablet replica
I20260809 15:50:47.361186 9552 raft_consensus.cc:2229] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [term 2 LEADER]: Raft consensus shutting down.
I20260809 15:50:47.362004 9552 raft_consensus.cc:2258] T 10055574bc224746bde901e394facf6a P e54dcbceb00649eb944818025b5a969f [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.363677 9552 tablet_replica.cc:330] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f: stopping tablet replica
I20260809 15:50:47.364156 9552 raft_consensus.cc:2229] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 15:50:47.364547 9552 raft_consensus.cc:2258] T 3e74d7900d9d485092a9e8b144fca6b0 P e54dcbceb00649eb944818025b5a969f [term 4 FOLLOWER]: Raft consensus is shut down!
W20260809 15:50:47.365677 9832 consensus_peers.cc:487] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 -> Peer e54dcbceb00649eb944818025b5a969f (127.9.84.2:42899): Couldn't send request to peer e54dcbceb00649eb944818025b5a969f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:50:47.395089 9552 tablet_server.cc:194] TabletServer@127.9.84.2:0 shutdown complete.
W20260809 15:50:47.415082 9830 consensus_peers.cc:487] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 -> Peer 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125): Couldn't send request to peer 042b068763ff48718439228aebb5cfcb. Status: Network error: Client connection negotiation failed: client connection to 127.9.84.1:35125: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 15:50:47.422322 9552 tablet_server.cc:178] TabletServer@127.9.84.3:0 shutting down...
W20260809 15:50:47.433617 9830 consensus_peers.cc:487] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 -> Peer 042b068763ff48718439228aebb5cfcb (127.9.84.1:35125): Couldn't send request to peer 042b068763ff48718439228aebb5cfcb. Status: Network error: Client connection negotiation failed: client connection to 127.9.84.1:35125: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 15:50:47.454476 9552 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260809 15:50:47.455737 9552 tablet_replica.cc:330] T 90c5ba0a4aea4aebaeecf735050a4528 P 4a0709a93f76476a8acbfa4e9eb955d7: stopping tablet replica
I20260809 15:50:47.457190 9552 raft_consensus.cc:2229] T 90c5ba0a4aea4aebaeecf735050a4528 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:50:47.457764 9552 raft_consensus.cc:2258] T 90c5ba0a4aea4aebaeecf735050a4528 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.459589 9552 tablet_replica.cc:330] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7: stopping tablet replica
I20260809 15:50:47.460058 9552 raft_consensus.cc:2229] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 15:50:47.460556 9552 raft_consensus.cc:2258] T 726df19d010d4f008eb75311543d0dd6 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.462149 9552 tablet_replica.cc:330] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7: stopping tablet replica
I20260809 15:50:47.462632 9552 raft_consensus.cc:2229] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 4 LEADER]: Raft consensus shutting down.
I20260809 15:50:47.463586 9552 raft_consensus.cc:2258] T 3e74d7900d9d485092a9e8b144fca6b0 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.465659 9552 tablet_replica.cc:330] T 10055574bc224746bde901e394facf6a P 4a0709a93f76476a8acbfa4e9eb955d7: stopping tablet replica
I20260809 15:50:47.466301 9552 raft_consensus.cc:2229] T 10055574bc224746bde901e394facf6a P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 15:50:47.466815 9552 raft_consensus.cc:2258] T 10055574bc224746bde901e394facf6a P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.468636 9552 tablet_replica.cc:330] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7: stopping tablet replica
I20260809 15:50:47.469193 9552 raft_consensus.cc:2229] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 LEADER]: Raft consensus shutting down.
I20260809 15:50:47.470249 9552 raft_consensus.cc:2258] T b7526b699d7d4b9680c8f2e98b9c72e2 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.471992 9552 tablet_replica.cc:330] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7: stopping tablet replica
I20260809 15:50:47.472479 9552 raft_consensus.cc:2229] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 LEADER]: Raft consensus shutting down.
I20260809 15:50:47.473492 9552 raft_consensus.cc:2258] T 691afd4416c84d7d8e595a5b7c5e7d4b P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.475160 9552 tablet_replica.cc:330] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7: stopping tablet replica
I20260809 15:50:47.475651 9552 raft_consensus.cc:2229] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 15:50:47.476104 9552 raft_consensus.cc:2258] T 0fa161c687aa4ee390cca51e913fe50f P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.477674 9552 tablet_replica.cc:330] T d9963b7dc24a41838578644fa0572566 P 4a0709a93f76476a8acbfa4e9eb955d7: stopping tablet replica
I20260809 15:50:47.478199 9552 raft_consensus.cc:2229] T d9963b7dc24a41838578644fa0572566 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 15:50:47.478680 9552 raft_consensus.cc:2258] T d9963b7dc24a41838578644fa0572566 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.480208 9552 tablet_replica.cc:330] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7: stopping tablet replica
I20260809 15:50:47.480625 9552 raft_consensus.cc:2229] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 15:50:47.481099 9552 raft_consensus.cc:2258] T a9d11b0e8c644e089dd34200345495c7 P 4a0709a93f76476a8acbfa4e9eb955d7 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.506942 9552 tablet_server.cc:194] TabletServer@127.9.84.3:0 shutdown complete.
I20260809 15:50:47.524333 9552 master.cc:537] Master@127.9.84.62:42749 shutting down...
I20260809 15:50:47.546370 9552 raft_consensus.cc:2229] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:50:47.547022 9552 raft_consensus.cc:2258] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:50:47.547359 9552 tablet_replica.cc:330] T 00000000000000000000000000000000 P de4c11a1f6c84b38afc547638849827b: stopping tablet replica
I20260809 15:50:47.564862 9552 master.cc:552] Master@127.9.84.62:42749 shutdown complete.
042b068763ff48718439228aebb5cfcb 4
4a0709a93f76476a8acbfa4e9eb955d7 1
e54dcbceb00649eb944818025b5a969f 4
042b068763ff48718439228aebb5cfcb 3
4a0709a93f76476a8acbfa4e9eb955d7 3
e54dcbceb00649eb944818025b5a969f 3
042b068763ff48718439228aebb5cfcb 0
4a0709a93f76476a8acbfa4e9eb955d7 1
e54dcbceb00649eb944818025b5a969f 8
042b068763ff48718439228aebb5cfcb 3
4a0709a93f76476a8acbfa4e9eb955d7 3
e54dcbceb00649eb944818025b5a969f 3
[ OK ] LeaderRebalancerTest.FunctionalTestForDivided (30320 ms)
[ RUN ] LeaderRebalancerTest.FunctionalTestForNotDivided
I20260809 15:50:47.607650 9552 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.84.62:37411
I20260809 15:50:47.608722 9552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:50:47.614523 10130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:50:47.614607 10129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:50:47.615267 10132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:50:47.616266 9552 server_base.cc:1034] running on GCE node
I20260809 15:50:47.617250 9552 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 15:50:47.617456 9552 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 15:50:47.617575 9552 hybrid_clock.cc:648] HybridClock initialized: now 1786290647617555 us; error 0 us; skew 500 ppm
I20260809 15:50:47.618011 9552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:50:47.623606 9552 webserver.cc:460] Webserver started at http://127.9.84.62:45145/ using document root <none> and password file <none>
I20260809 15:50:47.624025 9552 fs_manager.cc:359] Metadata directory not provided
I20260809 15:50:47.624176 9552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:50:47.624395 9552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:50:47.625584 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786290617080470-9552-0/minicluster-data/master-0-root/instance:
uuid: "9be32ac118be402fa78d175a5e66ff14"
format_stamp: "Formatted at 2026-08-09 15:50:47 on dist-test-slave-mn4r"
I20260809 15:50:47.629781 9552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 15:50:47.632776 10137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:50:47.633478 9552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:50:47.633738 9552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786290617080470-9552-0/minicluster-data/master-0-root
uuid: "9be32ac118be402fa78d175a5e66ff14"
format_stamp: "Formatted at 2026-08-09 15:50:47 on dist-test-slave-mn4r"
I20260809 15:50:47.634006 9552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786290617080470-9552-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786290617080470-9552-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786290617080470-9552-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:50:47.645824 9552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:50:47.647096 9552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:50:47.685281 9552 rpc_server.cc:307] RPC server started. Bound to: 127.9.84.62:37411
I20260809 15:50:47.685391 10200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.84.62:37411 every 8 connection(s)
I20260809 15:50:47.689128 10202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:47.698133 10202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14: Bootstrap starting.
I20260809 15:50:47.702565 10202 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:47.706557 10202 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14: No bootstrap required, opened a new log
I20260809 15:50:47.708350 10202 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be32ac118be402fa78d175a5e66ff14" member_type: VOTER }
I20260809 15:50:47.708802 10202 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:47.708969 10202 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9be32ac118be402fa78d175a5e66ff14, State: Initialized, Role: FOLLOWER
I20260809 15:50:47.709510 10202 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be32ac118be402fa78d175a5e66ff14" member_type: VOTER }
I20260809 15:50:47.709928 10202 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:50:47.710145 10202 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:50:47.710357 10202 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:47.713758 10202 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be32ac118be402fa78d175a5e66ff14" member_type: VOTER }
I20260809 15:50:47.714202 10202 leader_election.cc:302] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9be32ac118be402fa78d175a5e66ff14; no voters:
I20260809 15:50:47.715204 10202 leader_election.cc:288] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:50:47.715593 10205 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:47.716871 10205 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14 [term 1 LEADER]: Becoming Leader. State: Replica: 9be32ac118be402fa78d175a5e66ff14, State: Running, Role: LEADER
I20260809 15:50:47.717542 10205 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be32ac118be402fa78d175a5e66ff14" member_type: VOTER }
I20260809 15:50:47.718195 10202 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:50:47.723654 10206 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9be32ac118be402fa78d175a5e66ff14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be32ac118be402fa78d175a5e66ff14" member_type: VOTER } }
I20260809 15:50:47.723698 10207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9be32ac118be402fa78d175a5e66ff14. Latest consensus state: current_term: 1 leader_uuid: "9be32ac118be402fa78d175a5e66ff14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9be32ac118be402fa78d175a5e66ff14" member_type: VOTER } }
I20260809 15:50:47.724298 10206 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14 [sys.catalog]: This master's current role is: LEADER
I20260809 15:50:47.724314 10207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14 [sys.catalog]: This master's current role is: LEADER
I20260809 15:50:47.732583 10213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:50:47.738514 10213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:50:47.739529 9552 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260809 15:50:47.746526 10213 catalog_manager.cc:1347] Generated new cluster ID: 107a1eb245f844d08f3fe8fbf8985b1c
I20260809 15:50:47.746766 10213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:50:47.763871 10213 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:50:47.765455 10213 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:50:47.776914 10213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14: Generated new TSK 0
I20260809 15:50:47.777465 10213 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:50:47.805974 9552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:50:47.811933 10225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:50:47.812898 10226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:50:47.814658 10229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:50:47.815152 9552 server_base.cc:1034] running on GCE node
I20260809 15:50:47.816004 9552 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 15:50:47.816213 9552 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 15:50:47.816370 9552 hybrid_clock.cc:648] HybridClock initialized: now 1786290647816353 us; error 0 us; skew 500 ppm
I20260809 15:50:47.816876 9552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:50:47.819427 9552 webserver.cc:460] Webserver started at http://127.9.84.1:39825/ using document root <none> and password file <none>
I20260809 15:50:47.819938 9552 fs_manager.cc:359] Metadata directory not provided
I20260809 15:50:47.820183 9552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:50:47.820437 9552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:50:47.821499 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786290617080470-9552-0/minicluster-data/ts-0-root/instance:
uuid: "4932e5163456455ca4b8c5a7736031f4"
format_stamp: "Formatted at 2026-08-09 15:50:47 on dist-test-slave-mn4r"
I20260809 15:50:47.825830 9552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 15:50:47.828902 10234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:50:47.829756 9552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:50:47.830147 9552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786290617080470-9552-0/minicluster-data/ts-0-root
uuid: "4932e5163456455ca4b8c5a7736031f4"
format_stamp: "Formatted at 2026-08-09 15:50:47 on dist-test-slave-mn4r"
I20260809 15:50:47.830519 9552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786290617080470-9552-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786290617080470-9552-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786290617080470-9552-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:50:47.842029 9552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:50:47.843127 9552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:50:47.844439 9552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:50:47.846668 9552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:50:47.846841 9552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:50:47.847108 9552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:50:47.847283 9552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:50:47.887741 9552 rpc_server.cc:307] RPC server started. Bound to: 127.9.84.1:40561
I20260809 15:50:47.887861 10302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.84.1:40561 every 8 connection(s)
I20260809 15:50:47.892077 9552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:50:47.899717 10307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:50:47.902527 10308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:50:47.906965 9552 server_base.cc:1034] running on GCE node
W20260809 15:50:47.906755 10310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:50:47.908206 10303 heartbeater.cc:347] Connected to a master server at 127.9.84.62:37411
I20260809 15:50:47.908299 9552 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 15:50:47.908594 9552 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 15:50:47.908607 10303 heartbeater.cc:464] Registering TS with master...
I20260809 15:50:47.908874 9552 hybrid_clock.cc:648] HybridClock initialized: now 1786290647908857 us; error 0 us; skew 500 ppm
I20260809 15:50:47.909492 9552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:50:47.909459 10303 heartbeater.cc:511] Master 127.9.84.62:37411 requested a full tablet report, sending...
I20260809 15:50:47.911993 10158 ts_manager.cc:194] Registered new tserver with Master: 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561)
I20260809 15:50:47.912693 9552 webserver.cc:460] Webserver started at http://127.9.84.2:35419/ using document root <none> and password file <none>
I20260809 15:50:47.913348 9552 fs_manager.cc:359] Metadata directory not provided
I20260809 15:50:47.913599 9552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:50:47.913905 9552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:50:47.914726 10158 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43372
I20260809 15:50:47.915459 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786290617080470-9552-0/minicluster-data/ts-1-root/instance:
uuid: "df82df23ca924767bfc57e19c09e245a"
format_stamp: "Formatted at 2026-08-09 15:50:47 on dist-test-slave-mn4r"
I20260809 15:50:47.919869 9552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260809 15:50:47.922897 10315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:50:47.923607 9552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:50:47.923870 9552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786290617080470-9552-0/minicluster-data/ts-1-root
uuid: "df82df23ca924767bfc57e19c09e245a"
format_stamp: "Formatted at 2026-08-09 15:50:47 on dist-test-slave-mn4r"
I20260809 15:50:47.924146 9552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786290617080470-9552-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786290617080470-9552-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786290617080470-9552-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:50:47.939967 9552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:50:47.941020 9552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:50:47.942423 9552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:50:47.944434 9552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:50:47.944638 9552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:50:47.944890 9552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:50:47.945060 9552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:50:47.983373 9552 rpc_server.cc:307] RPC server started. Bound to: 127.9.84.2:35197
I20260809 15:50:47.983472 10386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.84.2:35197 every 8 connection(s)
I20260809 15:50:47.987933 9552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:50:47.994781 10390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:50:47.996032 10392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:50:48.002554 9552 server_base.cc:1034] running on GCE node
I20260809 15:50:48.003686 10387 heartbeater.cc:347] Connected to a master server at 127.9.84.62:37411
W20260809 15:50:48.003847 10395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:50:48.004109 10387 heartbeater.cc:464] Registering TS with master...
I20260809 15:50:48.004752 9552 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
I20260809 15:50:48.004837 10387 heartbeater.cc:511] Master 127.9.84.62:37411 requested a full tablet report, sending...
W20260809 15:50:48.005009 9552 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 15:50:48.005257 9552 hybrid_clock.cc:648] HybridClock initialized: now 1786290648005242 us; error 0 us; skew 500 ppm
I20260809 15:50:48.005936 9552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:50:48.006870 10158 ts_manager.cc:194] Registered new tserver with Master: df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197)
I20260809 15:50:48.008670 9552 webserver.cc:460] Webserver started at http://127.9.84.3:46021/ using document root <none> and password file <none>
I20260809 15:50:48.008735 10158 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43376
I20260809 15:50:48.009254 9552 fs_manager.cc:359] Metadata directory not provided
I20260809 15:50:48.009490 9552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:50:48.009859 9552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:50:48.011096 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786290617080470-9552-0/minicluster-data/ts-2-root/instance:
uuid: "972dcf60b2944faa9d0ceec67f064e5a"
format_stamp: "Formatted at 2026-08-09 15:50:48 on dist-test-slave-mn4r"
I20260809 15:50:48.015282 9552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 15:50:48.018261 10400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:50:48.019138 9552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260809 15:50:48.019374 9552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786290617080470-9552-0/minicluster-data/ts-2-root
uuid: "972dcf60b2944faa9d0ceec67f064e5a"
format_stamp: "Formatted at 2026-08-09 15:50:48 on dist-test-slave-mn4r"
I20260809 15:50:48.019662 9552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786290617080470-9552-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786290617080470-9552-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.FunctionalTestForNotDivided.1786290617080470-9552-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:50:48.050441 9552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:50:48.051532 9552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:50:48.052866 9552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:50:48.054970 9552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:50:48.055176 9552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:50:48.055435 9552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:50:48.055599 9552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:50:48.094340 9552 rpc_server.cc:307] RPC server started. Bound to: 127.9.84.3:44211
I20260809 15:50:48.094431 10471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.84.3:44211 every 8 connection(s)
I20260809 15:50:48.109272 10472 heartbeater.cc:347] Connected to a master server at 127.9.84.62:37411
I20260809 15:50:48.109668 10472 heartbeater.cc:464] Registering TS with master...
I20260809 15:50:48.110345 10472 heartbeater.cc:511] Master 127.9.84.62:37411 requested a full tablet report, sending...
I20260809 15:50:48.112123 10158 ts_manager.cc:194] Registered new tserver with Master: 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:48.113214 9552 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.015777912s
I20260809 15:50:48.113927 10158 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:43382
I20260809 15:50:48.115847 9552 test_util.cc:268] Using random seed: 864807537
I20260809 15:50:48.139556 10158 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:43398:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s""\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:50:48.142439 10158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:50:48.214733 10349 tablet_service.cc:1467] Processing CreateTablet for tablet 4c7503a4f3ba42d7b51a0fc1f1521c01 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260809 15:50:48.216127 10347 tablet_service.cc:1467] Processing CreateTablet for tablet 5ec90e88181c4e5085d12f183609cbae (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260809 15:50:48.217595 10263 tablet_service.cc:1467] Processing CreateTablet for tablet 4c7503a4f3ba42d7b51a0fc1f1521c01 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260809 15:50:48.218931 10263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c7503a4f3ba42d7b51a0fc1f1521c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.218596 10346 tablet_service.cc:1467] Processing CreateTablet for tablet b5cc2389e9a74343b1e06708819e63c2 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260809 15:50:48.219177 10262 tablet_service.cc:1467] Processing CreateTablet for tablet eb07891a9e804c51a61318118a0e8886 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260809 15:50:48.220577 10262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb07891a9e804c51a61318118a0e8886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.220623 10259 tablet_service.cc:1467] Processing CreateTablet for tablet b0af688bc961452abb333bd49c4c595d (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260809 15:50:48.220885 10345 tablet_service.cc:1467] Processing CreateTablet for tablet b0af688bc961452abb333bd49c4c595d (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260809 15:50:48.223299 10260 tablet_service.cc:1467] Processing CreateTablet for tablet b5cc2389e9a74343b1e06708819e63c2 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260809 15:50:48.223378 10348 tablet_service.cc:1467] Processing CreateTablet for tablet eb07891a9e804c51a61318118a0e8886 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260809 15:50:48.216169 10349 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c7503a4f3ba42d7b51a0fc1f1521c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.225515 10261 tablet_service.cc:1467] Processing CreateTablet for tablet 5ec90e88181c4e5085d12f183609cbae (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260809 15:50:48.227324 10258 tablet_service.cc:1467] Processing CreateTablet for tablet 00c2886e82e647e6986c2ace7f0fb5d2 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260809 15:50:48.221939 10259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0af688bc961452abb333bd49c4c595d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.229888 10261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ec90e88181c4e5085d12f183609cbae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.230628 10348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb07891a9e804c51a61318118a0e8886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.230993 10260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5cc2389e9a74343b1e06708819e63c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.231654 10258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00c2886e82e647e6986c2ace7f0fb5d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.232800 10346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5cc2389e9a74343b1e06708819e63c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.238562 10345 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0af688bc961452abb333bd49c4c595d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.240378 10347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ec90e88181c4e5085d12f183609cbae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.257674 10342 tablet_service.cc:1467] Processing CreateTablet for tablet 00c2886e82e647e6986c2ace7f0fb5d2 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260809 15:50:48.259023 10342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00c2886e82e647e6986c2ace7f0fb5d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.267047 10259 tablet_service.cc:1467] Processing CreateTablet for tablet a4ff800c08cd44158fc25d6fd4735dc2 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260809 15:50:48.267376 10341 tablet_service.cc:1467] Processing CreateTablet for tablet a4ff800c08cd44158fc25d6fd4735dc2 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260809 15:50:48.270005 10341 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4ff800c08cd44158fc25d6fd4735dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.271903 10490 tablet_bootstrap.cc:492] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4: Bootstrap starting.
I20260809 15:50:48.273854 10340 tablet_service.cc:1467] Processing CreateTablet for tablet b118cb45b2544ec68799e11793d4b481 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260809 15:50:48.275105 10340 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b118cb45b2544ec68799e11793d4b481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.276151 10262 tablet_service.cc:1467] Processing CreateTablet for tablet cc8769d170ef4bd089200728cd5f52cf (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260809 15:50:48.277153 10339 tablet_service.cc:1467] Processing CreateTablet for tablet cc8769d170ef4bd089200728cd5f52cf (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260809 15:50:48.278241 10490 tablet_bootstrap.cc:654] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.278398 10339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc8769d170ef4bd089200728cd5f52cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.280648 10258 tablet_service.cc:1467] Processing CreateTablet for tablet b31c1906f58b40d48decf694093d62a3 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260809 15:50:48.281620 10338 tablet_service.cc:1467] Processing CreateTablet for tablet b31c1906f58b40d48decf694093d62a3 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260809 15:50:48.287163 10338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b31c1906f58b40d48decf694093d62a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.292723 10433 tablet_service.cc:1467] Processing CreateTablet for tablet 4c7503a4f3ba42d7b51a0fc1f1521c01 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION VALUES < 214748364
I20260809 15:50:48.295229 10432 tablet_service.cc:1467] Processing CreateTablet for tablet eb07891a9e804c51a61318118a0e8886 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728
I20260809 15:50:48.295174 10425 tablet_service.cc:1467] Processing CreateTablet for tablet b118cb45b2544ec68799e11793d4b481 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260809 15:50:48.297336 10431 tablet_service.cc:1467] Processing CreateTablet for tablet 5ec90e88181c4e5085d12f183609cbae (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092
I20260809 15:50:48.297737 10424 tablet_service.cc:1467] Processing CreateTablet for tablet cc8769d170ef4bd089200728cd5f52cf (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276
I20260809 15:50:48.291006 10260 tablet_service.cc:1467] Processing CreateTablet for tablet b118cb45b2544ec68799e11793d4b481 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912
I20260809 15:50:48.293941 10433 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c7503a4f3ba42d7b51a0fc1f1521c01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.299612 10424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc8769d170ef4bd089200728cd5f52cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.307513 10428 tablet_service.cc:1467] Processing CreateTablet for tablet b0af688bc961452abb333bd49c4c595d (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820
I20260809 15:50:48.310374 10425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b118cb45b2544ec68799e11793d4b481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.310310 10426 tablet_service.cc:1467] Processing CreateTablet for tablet a4ff800c08cd44158fc25d6fd4735dc2 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548
I20260809 15:50:48.312047 10426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4ff800c08cd44158fc25d6fd4735dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.314442 10431 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ec90e88181c4e5085d12f183609cbae. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.315768 10430 tablet_service.cc:1467] Processing CreateTablet for tablet b5cc2389e9a74343b1e06708819e63c2 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456
I20260809 15:50:48.317507 10430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5cc2389e9a74343b1e06708819e63c2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.318544 10432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb07891a9e804c51a61318118a0e8886. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.320544 10262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cc8769d170ef4bd089200728cd5f52cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.326839 10260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b118cb45b2544ec68799e11793d4b481. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.331135 10258 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b31c1906f58b40d48decf694093d62a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.305439 10427 tablet_service.cc:1467] Processing CreateTablet for tablet 00c2886e82e647e6986c2ace7f0fb5d2 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184
I20260809 15:50:48.333945 10428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0af688bc961452abb333bd49c4c595d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.340365 10427 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00c2886e82e647e6986c2ace7f0fb5d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.344125 10259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a4ff800c08cd44158fc25d6fd4735dc2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.365345 10423 tablet_service.cc:1467] Processing CreateTablet for tablet b31c1906f58b40d48decf694093d62a3 (DEFAULT_TABLE table=test-workload [id=a8256bb7099142dc9a37cc637daeb99f]), partition=RANGE (key) PARTITION 1932735276 <= VALUES
I20260809 15:50:48.366674 10423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b31c1906f58b40d48decf694093d62a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:50:48.394104 10493 tablet_bootstrap.cc:492] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a: Bootstrap starting.
I20260809 15:50:48.401789 10495 tablet_bootstrap.cc:492] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a: Bootstrap starting.
I20260809 15:50:48.408051 10490 tablet_bootstrap.cc:492] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4: No bootstrap required, opened a new log
I20260809 15:50:48.408777 10490 ts_tablet_manager.cc:1397] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4: Time spent bootstrapping tablet: real 0.137s user 0.034s sys 0.043s
I20260809 15:50:48.411353 10490 raft_consensus.cc:348] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.412070 10490 raft_consensus.cc:374] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.413040 10490 raft_consensus.cc:729] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.413790 10490 consensus_queue.cc:260] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.416867 10493 tablet_bootstrap.cc:654] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.417821 10495 tablet_bootstrap.cc:654] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.431466 10490 ts_tablet_manager.cc:1428] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4: Time spent starting tablet: real 0.022s user 0.008s sys 0.008s
I20260809 15:50:48.432374 10303 heartbeater.cc:503] Master 127.9.84.62:37411 was elected leader, sending a full tablet report...
I20260809 15:50:48.434211 10493 tablet_bootstrap.cc:492] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a: No bootstrap required, opened a new log
I20260809 15:50:48.434672 10493 ts_tablet_manager.cc:1397] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a: Time spent bootstrapping tablet: real 0.041s user 0.006s sys 0.010s
I20260809 15:50:48.434775 10490 tablet_bootstrap.cc:492] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4: Bootstrap starting.
I20260809 15:50:48.437096 10493 raft_consensus.cc:348] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.437785 10493 raft_consensus.cc:374] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.438053 10493 raft_consensus.cc:729] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.440482 10490 tablet_bootstrap.cc:654] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.441180 10495 tablet_bootstrap.cc:492] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a: No bootstrap required, opened a new log
I20260809 15:50:48.441656 10495 ts_tablet_manager.cc:1397] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a: Time spent bootstrapping tablet: real 0.041s user 0.012s sys 0.003s
I20260809 15:50:48.444648 10495 raft_consensus.cc:348] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.445578 10495 raft_consensus.cc:374] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.445938 10495 raft_consensus.cc:729] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.446738 10493 consensus_queue.cc:260] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.447145 10495 consensus_queue.cc:260] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.449456 10387 heartbeater.cc:503] Master 127.9.84.62:37411 was elected leader, sending a full tablet report...
I20260809 15:50:48.449934 10493 ts_tablet_manager.cc:1428] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a: Time spent starting tablet: real 0.015s user 0.002s sys 0.004s
I20260809 15:50:48.450796 10493 tablet_bootstrap.cc:492] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a: Bootstrap starting.
I20260809 15:50:48.456205 10472 heartbeater.cc:503] Master 127.9.84.62:37411 was elected leader, sending a full tablet report...
I20260809 15:50:48.457131 10495 ts_tablet_manager.cc:1428] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a: Time spent starting tablet: real 0.015s user 0.008s sys 0.007s
I20260809 15:50:48.457991 10495 tablet_bootstrap.cc:492] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a: Bootstrap starting.
I20260809 15:50:48.459079 10490 tablet_bootstrap.cc:492] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4: No bootstrap required, opened a new log
I20260809 15:50:48.459517 10490 ts_tablet_manager.cc:1397] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4: Time spent bootstrapping tablet: real 0.025s user 0.009s sys 0.004s
I20260809 15:50:48.461992 10490 raft_consensus.cc:348] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.462762 10490 raft_consensus.cc:374] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.463070 10490 raft_consensus.cc:729] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.463706 10490 consensus_queue.cc:260] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.464910 10495 tablet_bootstrap.cc:654] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.467240 10490 ts_tablet_manager.cc:1428] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260809 15:50:48.468252 10490 tablet_bootstrap.cc:492] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4: Bootstrap starting.
I20260809 15:50:48.474036 10493 tablet_bootstrap.cc:654] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.475294 10490 tablet_bootstrap.cc:654] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.476213 10495 tablet_bootstrap.cc:492] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a: No bootstrap required, opened a new log
I20260809 15:50:48.476691 10495 ts_tablet_manager.cc:1397] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.008s
I20260809 15:50:48.479671 10495 raft_consensus.cc:348] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.480587 10495 raft_consensus.cc:374] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.480928 10495 raft_consensus.cc:729] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.481973 10495 consensus_queue.cc:260] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.485009 10495 ts_tablet_manager.cc:1428] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260809 15:50:48.486107 10495 tablet_bootstrap.cc:492] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a: Bootstrap starting.
I20260809 15:50:48.491842 10493 tablet_bootstrap.cc:492] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a: No bootstrap required, opened a new log
I20260809 15:50:48.492362 10493 ts_tablet_manager.cc:1397] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a: Time spent bootstrapping tablet: real 0.042s user 0.006s sys 0.005s
I20260809 15:50:48.493600 10495 tablet_bootstrap.cc:654] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.495107 10493 raft_consensus.cc:348] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.495915 10493 raft_consensus.cc:374] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.496281 10493 raft_consensus.cc:729] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.496583 10490 tablet_bootstrap.cc:492] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4: No bootstrap required, opened a new log
I20260809 15:50:48.497169 10490 ts_tablet_manager.cc:1397] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4: Time spent bootstrapping tablet: real 0.029s user 0.021s sys 0.007s
I20260809 15:50:48.497097 10493 consensus_queue.cc:260] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.499644 10493 ts_tablet_manager.cc:1428] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260809 15:50:48.500566 10493 tablet_bootstrap.cc:492] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a: Bootstrap starting.
I20260809 15:50:48.500422 10490 raft_consensus.cc:348] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.501436 10490 raft_consensus.cc:374] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.501811 10490 raft_consensus.cc:729] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.503326 10490 consensus_queue.cc:260] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.505895 10495 tablet_bootstrap.cc:492] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a: No bootstrap required, opened a new log
I20260809 15:50:48.506325 10495 ts_tablet_manager.cc:1397] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a: Time spent bootstrapping tablet: real 0.020s user 0.017s sys 0.002s
I20260809 15:50:48.507277 10493 tablet_bootstrap.cc:654] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.508800 10495 raft_consensus.cc:348] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.509501 10495 raft_consensus.cc:374] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.509779 10495 raft_consensus.cc:729] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.510607 10495 consensus_queue.cc:260] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.511600 10490 ts_tablet_manager.cc:1428] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4: Time spent starting tablet: real 0.014s user 0.004s sys 0.004s
I20260809 15:50:48.512719 10495 ts_tablet_manager.cc:1428] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a: Time spent starting tablet: real 0.006s user 0.003s sys 0.001s
I20260809 15:50:48.513834 10495 tablet_bootstrap.cc:492] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a: Bootstrap starting.
I20260809 15:50:48.514760 10490 tablet_bootstrap.cc:492] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4: Bootstrap starting.
I20260809 15:50:48.519584 10495 tablet_bootstrap.cc:654] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.520769 10490 tablet_bootstrap.cc:654] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.529327 10493 tablet_bootstrap.cc:492] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a: No bootstrap required, opened a new log
I20260809 15:50:48.529755 10493 ts_tablet_manager.cc:1397] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a: Time spent bootstrapping tablet: real 0.029s user 0.011s sys 0.013s
I20260809 15:50:48.532047 10490 tablet_bootstrap.cc:492] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4: No bootstrap required, opened a new log
I20260809 15:50:48.532506 10490 ts_tablet_manager.cc:1397] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4: Time spent bootstrapping tablet: real 0.018s user 0.009s sys 0.000s
I20260809 15:50:48.532855 10495 tablet_bootstrap.cc:492] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a: No bootstrap required, opened a new log
I20260809 15:50:48.532418 10493 raft_consensus.cc:348] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.533203 10493 raft_consensus.cc:374] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.533324 10495 ts_tablet_manager.cc:1397] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a: Time spent bootstrapping tablet: real 0.020s user 0.015s sys 0.000s
I20260809 15:50:48.533507 10493 raft_consensus.cc:729] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.534235 10493 consensus_queue.cc:260] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.535120 10490 raft_consensus.cc:348] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.535841 10490 raft_consensus.cc:374] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.536101 10490 raft_consensus.cc:729] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.535949 10495 raft_consensus.cc:348] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.536650 10495 raft_consensus.cc:374] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.536963 10495 raft_consensus.cc:729] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.536847 10490 consensus_queue.cc:260] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.537784 10495 consensus_queue.cc:260] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.539726 10490 ts_tablet_manager.cc:1428] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4: Time spent starting tablet: real 0.007s user 0.004s sys 0.003s
I20260809 15:50:48.540943 10493 ts_tablet_manager.cc:1428] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a: Time spent starting tablet: real 0.011s user 0.004s sys 0.005s
I20260809 15:50:48.541826 10493 tablet_bootstrap.cc:492] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a: Bootstrap starting.
I20260809 15:50:48.542999 10503 raft_consensus.cc:482] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:48.543457 10503 raft_consensus.cc:504] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.547801 10493 tablet_bootstrap.cc:654] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.550256 10503 leader_election.cc:288] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197)
I20260809 15:50:48.554489 10496 raft_consensus.cc:482] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:48.555042 10496 raft_consensus.cc:504] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.557156 10495 ts_tablet_manager.cc:1428] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a: Time spent starting tablet: real 0.023s user 0.007s sys 0.000s
I20260809 15:50:48.560912 10490 tablet_bootstrap.cc:492] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4: Bootstrap starting.
I20260809 15:50:48.566161 10495 tablet_bootstrap.cc:492] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a: Bootstrap starting.
I20260809 15:50:48.567611 10490 tablet_bootstrap.cc:654] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.571545 10496 leader_election.cc:288] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:48.572963 10495 tablet_bootstrap.cc:654] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.577813 10503 raft_consensus.cc:482] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:48.578330 10503 raft_consensus.cc:504] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.580869 10501 raft_consensus.cc:482] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:48.581094 10503 leader_election.cc:288] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197)
I20260809 15:50:48.581492 10501 raft_consensus.cc:504] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.586164 10501 leader_election.cc:288] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:48.614404 10275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00c2886e82e647e6986c2ace7f0fb5d2" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e5163456455ca4b8c5a7736031f4" is_pre_election: true
I20260809 15:50:48.615413 10275 raft_consensus.cc:2454] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 0.
I20260809 15:50:48.616904 10274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc8769d170ef4bd089200728cd5f52cf" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e5163456455ca4b8c5a7736031f4" is_pre_election: true
I20260809 15:50:48.617090 10401 leader_election.cc:302] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, 972dcf60b2944faa9d0ceec67f064e5a; no voters:
I20260809 15:50:48.618253 10503 raft_consensus.cc:2781] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:50:48.618687 10503 raft_consensus.cc:482] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:50:48.619264 10503 raft_consensus.cc:3037] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:50:48.618631 10401 leader_election.cc:341] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:48.629367 10503 raft_consensus.cc:504] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.667491 10274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00c2886e82e647e6986c2ace7f0fb5d2" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
I20260809 15:50:48.668289 10274 raft_consensus.cc:3037] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:48.677624 10274 raft_consensus.cc:2454] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 1.
I20260809 15:50:48.693001 10401 leader_election.cc:302] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, 972dcf60b2944faa9d0ceec67f064e5a; no voters:
I20260809 15:50:48.678646 10503 leader_election.cc:288] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 election: Requested vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197)
I20260809 15:50:48.703615 10503 raft_consensus.cc:2781] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:48.704191 10503 raft_consensus.cc:686] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Becoming Leader. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Running, Role: LEADER
I20260809 15:50:48.708951 10361 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00c2886e82e647e6986c2ace7f0fb5d2" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a" is_pre_election: true
I20260809 15:50:48.710451 10360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc8769d170ef4bd089200728cd5f52cf" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a" is_pre_election: true
I20260809 15:50:48.711846 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00c2886e82e647e6986c2ace7f0fb5d2" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:50:48.712157 10521 raft_consensus.cc:482] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:48.712603 10521 raft_consensus.cc:504] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
W20260809 15:50:48.713164 10406 leader_election.cc:341] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197): Illegal state: must be running to vote when last-logged opid is not known
W20260809 15:50:48.714174 10406 leader_election.cc:341] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:48.714547 10501 raft_consensus.cc:482] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:48.714684 10406 leader_election.cc:302] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 972dcf60b2944faa9d0ceec67f064e5a; no voters: 4932e5163456455ca4b8c5a7736031f4, df82df23ca924767bfc57e19c09e245a
I20260809 15:50:48.714980 10501 raft_consensus.cc:504] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.719635 10501 leader_election.cc:288] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:48.723270 10495 tablet_bootstrap.cc:492] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a: No bootstrap required, opened a new log
I20260809 15:50:48.723856 10495 ts_tablet_manager.cc:1397] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a: Time spent bootstrapping tablet: real 0.158s user 0.026s sys 0.046s
I20260809 15:50:48.705245 10503 consensus_queue.cc:237] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
W20260809 15:50:48.731184 10216 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 15:50:48.731808 10495 raft_consensus.cc:348] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.732740 10495 raft_consensus.cc:374] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.733090 10495 raft_consensus.cc:729] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.735255 10521 leader_election.cc:288] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197)
I20260809 15:50:48.735878 10521 raft_consensus.cc:2726] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 15:50:48.736969 10406 leader_election.cc:341] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:48.738466 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b31c1906f58b40d48decf694093d62a3" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a" is_pre_election: true
I20260809 15:50:48.739282 10490 tablet_bootstrap.cc:492] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4: No bootstrap required, opened a new log
I20260809 15:50:48.739749 10490 ts_tablet_manager.cc:1397] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4: Time spent bootstrapping tablet: real 0.179s user 0.075s sys 0.074s
I20260809 15:50:48.741114 10495 consensus_queue.cc:260] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
W20260809 15:50:48.740535 10406 leader_election.cc:341] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:48.740455 10445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a" is_pre_election: true
I20260809 15:50:48.743260 10274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b31c1906f58b40d48decf694093d62a3" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e5163456455ca4b8c5a7736031f4" is_pre_election: true
I20260809 15:50:48.743465 10493 tablet_bootstrap.cc:492] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a: No bootstrap required, opened a new log
I20260809 15:50:48.741451 10444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ec90e88181c4e5085d12f183609cbae" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a" is_pre_election: true
I20260809 15:50:48.743976 10493 ts_tablet_manager.cc:1397] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a: Time spent bootstrapping tablet: real 0.202s user 0.076s sys 0.102s
W20260809 15:50:48.744575 10401 leader_election.cc:341] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561): Illegal state: must be running to vote when last-logged opid is not known
W20260809 15:50:48.745009 10321 leader_election.cc:341] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:48.743480 10495 ts_tablet_manager.cc:1428] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a: Time spent starting tablet: real 0.019s user 0.008s sys 0.000s
I20260809 15:50:48.744781 10490 raft_consensus.cc:348] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.744987 10401 leader_election.cc:302] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 972dcf60b2944faa9d0ceec67f064e5a; no voters: 4932e5163456455ca4b8c5a7736031f4, df82df23ca924767bfc57e19c09e245a
W20260809 15:50:48.746377 10321 leader_election.cc:341] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:48.746902 10490 raft_consensus.cc:374] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.746690 10493 raft_consensus.cc:348] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.747323 10490 raft_consensus.cc:729] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.747365 10493 raft_consensus.cc:374] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.747272 10495 tablet_bootstrap.cc:492] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a: Bootstrap starting.
I20260809 15:50:48.746930 10522 raft_consensus.cc:2726] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:50:48.747898 10493 raft_consensus.cc:729] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.748428 10490 consensus_queue.cc:260] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.748731 10493 consensus_queue.cc:260] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.749092 10444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0af688bc961452abb333bd49c4c595d" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a" is_pre_election: true
W20260809 15:50:48.750401 10237 leader_election.cc:341] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:48.752966 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0af688bc961452abb333bd49c4c595d" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a" is_pre_election: true
I20260809 15:50:48.753767 10495 tablet_bootstrap.cc:654] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.753752 10490 ts_tablet_manager.cc:1428] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
W20260809 15:50:48.754427 10238 leader_election.cc:341] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:48.754992 10490 tablet_bootstrap.cc:492] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4: Bootstrap starting.
I20260809 15:50:48.754868 10238 leader_election.cc:302] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4; no voters: 972dcf60b2944faa9d0ceec67f064e5a, df82df23ca924767bfc57e19c09e245a
I20260809 15:50:48.755954 10496 raft_consensus.cc:2726] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:50:48.759433 10496 raft_consensus.cc:482] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:48.759873 10496 raft_consensus.cc:504] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.761775 10496 leader_election.cc:288] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:48.763084 10490 tablet_bootstrap.cc:654] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.763577 10444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb07891a9e804c51a61318118a0e8886" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a" is_pre_election: true
I20260809 15:50:48.764859 10495 tablet_bootstrap.cc:492] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a: No bootstrap required, opened a new log
I20260809 15:50:48.764725 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb07891a9e804c51a61318118a0e8886" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a" is_pre_election: true
I20260809 15:50:48.765408 10495 ts_tablet_manager.cc:1397] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a: Time spent bootstrapping tablet: real 0.018s user 0.016s sys 0.000s
W20260809 15:50:48.765479 10237 leader_election.cc:341] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211): Illegal state: must be running to vote when last-logged opid is not known
W20260809 15:50:48.766021 10238 leader_election.cc:341] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:48.766495 10238 leader_election.cc:302] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4; no voters: 972dcf60b2944faa9d0ceec67f064e5a, df82df23ca924767bfc57e19c09e245a
I20260809 15:50:48.767439 10493 ts_tablet_manager.cc:1428] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a: Time spent starting tablet: real 0.023s user 0.006s sys 0.000s
I20260809 15:50:48.767479 10496 raft_consensus.cc:2726] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:50:48.768457 10495 raft_consensus.cc:348] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.769325 10495 raft_consensus.cc:374] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.769608 10495 raft_consensus.cc:729] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.770309 10495 consensus_queue.cc:260] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.772138 10495 ts_tablet_manager.cc:1428] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260809 15:50:48.772820 10495 tablet_bootstrap.cc:492] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a: Bootstrap starting.
I20260809 15:50:48.777824 10274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e5163456455ca4b8c5a7736031f4" is_pre_election: true
I20260809 15:50:48.778167 10275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ec90e88181c4e5085d12f183609cbae" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e5163456455ca4b8c5a7736031f4" is_pre_election: true
I20260809 15:50:48.778614 10495 tablet_bootstrap.cc:654] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.778491 10274 raft_consensus.cc:2454] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df82df23ca924767bfc57e19c09e245a in term 0.
I20260809 15:50:48.778978 10493 tablet_bootstrap.cc:492] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a: Bootstrap starting.
W20260809 15:50:48.779997 10316 leader_election.cc:341] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:48.780452 10316 leader_election.cc:302] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df82df23ca924767bfc57e19c09e245a; no voters: 4932e5163456455ca4b8c5a7736031f4, 972dcf60b2944faa9d0ceec67f064e5a
I20260809 15:50:48.781698 10316 leader_election.cc:302] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, df82df23ca924767bfc57e19c09e245a; no voters: 972dcf60b2944faa9d0ceec67f064e5a
I20260809 15:50:48.782733 10501 raft_consensus.cc:2726] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:50:48.783278 10501 raft_consensus.cc:2781] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:50:48.783576 10501 raft_consensus.cc:482] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:50:48.783804 10493 tablet_bootstrap.cc:654] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.783883 10501 raft_consensus.cc:3037] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:48.784157 10495 tablet_bootstrap.cc:492] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a: No bootstrap required, opened a new log
I20260809 15:50:48.784569 10495 ts_tablet_manager.cc:1397] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260809 15:50:48.787184 10495 raft_consensus.cc:348] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.788072 10493 tablet_bootstrap.cc:492] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a: No bootstrap required, opened a new log
I20260809 15:50:48.787985 10495 raft_consensus.cc:374] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.788372 10495 raft_consensus.cc:729] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.788534 10493 ts_tablet_manager.cc:1397] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.002s
I20260809 15:50:48.789170 10495 consensus_queue.cc:260] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.789834 10503 raft_consensus.cc:482] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:48.786952 10157 catalog_manager.cc:5495] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3). New cstate: current_term: 1 leader_uuid: "972dcf60b2944faa9d0ceec67f064e5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: HEALTHY } } }
I20260809 15:50:48.790325 10503 raft_consensus.cc:504] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.791440 10495 ts_tablet_manager.cc:1428] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 15:50:48.791175 10493 raft_consensus.cc:348] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.791896 10493 raft_consensus.cc:374] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.792178 10493 raft_consensus.cc:729] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.792338 10495 tablet_bootstrap.cc:492] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a: Bootstrap starting.
I20260809 15:50:48.792308 10503 leader_election.cc:288] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197)
I20260809 15:50:48.793103 10274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5cc2389e9a74343b1e06708819e63c2" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e5163456455ca4b8c5a7736031f4" is_pre_election: true
I20260809 15:50:48.792845 10493 consensus_queue.cc:260] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.793740 10274 raft_consensus.cc:2454] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 0.
I20260809 15:50:48.794178 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5cc2389e9a74343b1e06708819e63c2" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a" is_pre_election: true
I20260809 15:50:48.794775 10358 raft_consensus.cc:2454] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 0.
I20260809 15:50:48.795073 10493 ts_tablet_manager.cc:1428] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a: Time spent starting tablet: real 0.006s user 0.002s sys 0.006s
I20260809 15:50:48.795010 10401 leader_election.cc:302] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, 972dcf60b2944faa9d0ceec67f064e5a; no voters:
I20260809 15:50:48.796358 10503 raft_consensus.cc:2781] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:50:48.796686 10503 raft_consensus.cc:482] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:50:48.797008 10503 raft_consensus.cc:3037] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:48.797851 10493 tablet_bootstrap.cc:492] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a: Bootstrap starting.
I20260809 15:50:48.798626 10495 tablet_bootstrap.cc:654] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.800527 10501 raft_consensus.cc:504] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.804070 10274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
I20260809 15:50:48.804620 10274 raft_consensus.cc:3037] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:48.808549 10493 tablet_bootstrap.cc:654] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.810909 10274 raft_consensus.cc:2454] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df82df23ca924767bfc57e19c09e245a in term 1.
I20260809 15:50:48.812790 10490 tablet_bootstrap.cc:492] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4: No bootstrap required, opened a new log
I20260809 15:50:48.813373 10490 ts_tablet_manager.cc:1397] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4: Time spent bootstrapping tablet: real 0.059s user 0.010s sys 0.002s
I20260809 15:50:48.813809 10503 raft_consensus.cc:504] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.815970 10490 raft_consensus.cc:348] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.816655 10490 raft_consensus.cc:374] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.816582 10275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5cc2389e9a74343b1e06708819e63c2" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
I20260809 15:50:48.816978 10490 raft_consensus.cc:729] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.817201 10275 raft_consensus.cc:3037] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:48.818029 10490 consensus_queue.cc:260] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.820814 10490 ts_tablet_manager.cc:1428] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260809 15:50:48.821915 10490 tablet_bootstrap.cc:492] T cc8769d170ef4bd089200728cd5f52cf P 4932e5163456455ca4b8c5a7736031f4: Bootstrap starting.
I20260809 15:50:48.822885 10316 leader_election.cc:302] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:50:48.823832 10528 raft_consensus.cc:2781] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:48.824864 10444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
W20260809 15:50:48.826295 10321 leader_election.cc:341] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:48.826905 10501 leader_election.cc:288] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 1 election: Requested vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:48.827456 10275 raft_consensus.cc:2454] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 1.
I20260809 15:50:48.828467 10401 leader_election.cc:302] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, 972dcf60b2944faa9d0ceec67f064e5a; no voters:
I20260809 15:50:48.831380 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5cc2389e9a74343b1e06708819e63c2" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:50:48.832049 10358 raft_consensus.cc:3037] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:48.837705 10490 tablet_bootstrap.cc:654] T cc8769d170ef4bd089200728cd5f52cf P 4932e5163456455ca4b8c5a7736031f4: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.839807 10358 raft_consensus.cc:2454] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 1.
I20260809 15:50:48.841424 10521 raft_consensus.cc:2781] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:48.842207 10521 raft_consensus.cc:686] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Becoming Leader. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Running, Role: LEADER
I20260809 15:50:48.843114 10521 consensus_queue.cc:237] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.845046 10503 leader_election.cc:288] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 election: Requested vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197)
I20260809 15:50:48.848719 10490 tablet_bootstrap.cc:492] T cc8769d170ef4bd089200728cd5f52cf P 4932e5163456455ca4b8c5a7736031f4: No bootstrap required, opened a new log
I20260809 15:50:48.849175 10490 ts_tablet_manager.cc:1397] T cc8769d170ef4bd089200728cd5f52cf P 4932e5163456455ca4b8c5a7736031f4: Time spent bootstrapping tablet: real 0.027s user 0.010s sys 0.002s
I20260809 15:50:48.852262 10490 raft_consensus.cc:348] T cc8769d170ef4bd089200728cd5f52cf P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.854099 10490 raft_consensus.cc:374] T cc8769d170ef4bd089200728cd5f52cf P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.854521 10490 raft_consensus.cc:729] T cc8769d170ef4bd089200728cd5f52cf P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.855014 10528 raft_consensus.cc:686] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 1 LEADER]: Becoming Leader. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Running, Role: LEADER
I20260809 15:50:48.855381 10490 consensus_queue.cc:260] T cc8769d170ef4bd089200728cd5f52cf P 4932e5163456455ca4b8c5a7736031f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.856101 10528 consensus_queue.cc:237] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.858144 10490 ts_tablet_manager.cc:1428] T cc8769d170ef4bd089200728cd5f52cf P 4932e5163456455ca4b8c5a7736031f4: Time spent starting tablet: real 0.009s user 0.004s sys 0.003s
I20260809 15:50:48.859170 10490 tablet_bootstrap.cc:492] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4: Bootstrap starting.
I20260809 15:50:48.861554 10503 raft_consensus.cc:482] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:48.862031 10503 raft_consensus.cc:504] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.864825 10275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0af688bc961452abb333bd49c4c595d" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e5163456455ca4b8c5a7736031f4" is_pre_election: true
I20260809 15:50:48.865486 10275 raft_consensus.cc:2454] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 0.
I20260809 15:50:48.866674 10490 tablet_bootstrap.cc:654] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.866521 10401 leader_election.cc:302] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, 972dcf60b2944faa9d0ceec67f064e5a; no voters:
I20260809 15:50:48.867421 10522 raft_consensus.cc:2781] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:50:48.867765 10522 raft_consensus.cc:482] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:50:48.868048 10522 raft_consensus.cc:3037] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:48.868809 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0af688bc961452abb333bd49c4c595d" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a" is_pre_election: true
W20260809 15:50:48.869951 10406 leader_election.cc:341] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:48.867452 10534 raft_consensus.cc:482] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:48.870922 10534 raft_consensus.cc:504] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.873528 10275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc8769d170ef4bd089200728cd5f52cf" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e5163456455ca4b8c5a7736031f4" is_pre_election: true
I20260809 15:50:48.874116 10275 raft_consensus.cc:2454] T cc8769d170ef4bd089200728cd5f52cf P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df82df23ca924767bfc57e19c09e245a in term 0.
I20260809 15:50:48.875072 10316 leader_election.cc:302] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:50:48.874946 10522 raft_consensus.cc:504] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.875803 10501 raft_consensus.cc:2781] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:50:48.876148 10501 raft_consensus.cc:482] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:50:48.876467 10501 raft_consensus.cc:3037] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:48.877849 10274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0af688bc961452abb333bd49c4c595d" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
I20260809 15:50:48.878407 10274 raft_consensus.cc:3037] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:48.882715 10501 raft_consensus.cc:504] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.885492 10273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc8769d170ef4bd089200728cd5f52cf" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
I20260809 15:50:48.886119 10273 raft_consensus.cc:3037] T cc8769d170ef4bd089200728cd5f52cf P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:48.886248 10501 leader_election.cc:288] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 1 election: Requested vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:48.889416 10503 leader_election.cc:288] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197)
I20260809 15:50:48.890436 10444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc8769d170ef4bd089200728cd5f52cf" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
I20260809 15:50:48.891014 10444 raft_consensus.cc:3037] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:48.894003 10534 leader_election.cc:288] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:48.894241 10445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cc8769d170ef4bd089200728cd5f52cf" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a" is_pre_election: true
I20260809 15:50:48.901611 10493 tablet_bootstrap.cc:492] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a: No bootstrap required, opened a new log
I20260809 15:50:48.901672 10444 raft_consensus.cc:2454] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df82df23ca924767bfc57e19c09e245a in term 1.
I20260809 15:50:48.901971 10495 tablet_bootstrap.cc:492] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a: No bootstrap required, opened a new log
I20260809 15:50:48.907814 10495 ts_tablet_manager.cc:1397] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a: Time spent bootstrapping tablet: real 0.116s user 0.027s sys 0.080s
I20260809 15:50:48.908202 10321 leader_election.cc:302] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 972dcf60b2944faa9d0ceec67f064e5a, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:50:48.909085 10528 raft_consensus.cc:2781] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:48.909548 10528 raft_consensus.cc:686] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [term 1 LEADER]: Becoming Leader. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Running, Role: LEADER
I20260809 15:50:48.910235 10528 consensus_queue.cc:237] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.910202 10495 raft_consensus.cc:348] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.910983 10495 raft_consensus.cc:374] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.911280 10495 raft_consensus.cc:729] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.911972 10495 consensus_queue.cc:260] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.902199 10493 ts_tablet_manager.cc:1397] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a: Time spent bootstrapping tablet: real 0.105s user 0.032s sys 0.044s
I20260809 15:50:48.914068 10495 ts_tablet_manager.cc:1428] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 15:50:48.914945 10495 tablet_bootstrap.cc:492] T 5ec90e88181c4e5085d12f183609cbae P 972dcf60b2944faa9d0ceec67f064e5a: Bootstrap starting.
I20260809 15:50:48.915194 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0af688bc961452abb333bd49c4c595d" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:50:48.914674 10158 catalog_manager.cc:5495] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a reported cstate change: term changed from 0 to 1, leader changed from <none> to df82df23ca924767bfc57e19c09e245a (127.9.84.2). New cstate: current_term: 1 leader_uuid: "df82df23ca924767bfc57e19c09e245a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:48.907080 10274 raft_consensus.cc:2454] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 1.
I20260809 15:50:48.916606 10401 leader_election.cc:302] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, 972dcf60b2944faa9d0ceec67f064e5a; no voters:
I20260809 15:50:48.906030 10273 raft_consensus.cc:2454] T cc8769d170ef4bd089200728cd5f52cf P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df82df23ca924767bfc57e19c09e245a in term 1.
I20260809 15:50:48.918253 10522 leader_election.cc:288] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 election: Requested vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197)
I20260809 15:50:48.920114 10522 raft_consensus.cc:2781] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:48.920702 10522 raft_consensus.cc:686] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Becoming Leader. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Running, Role: LEADER
W20260809 15:50:48.921805 10406 leader_election.cc:341] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:50:48.921479 10522 consensus_queue.cc:237] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.922483 10495 tablet_bootstrap.cc:654] T 5ec90e88181c4e5085d12f183609cbae P 972dcf60b2944faa9d0ceec67f064e5a: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.924103 10493 raft_consensus.cc:348] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.924888 10493 raft_consensus.cc:374] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.925271 10493 raft_consensus.cc:729] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.926321 10493 consensus_queue.cc:260] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.928615 10157 catalog_manager.cc:5495] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3). New cstate: current_term: 1 leader_uuid: "972dcf60b2944faa9d0ceec67f064e5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: HEALTHY } } }
I20260809 15:50:48.929757 10493 ts_tablet_manager.cc:1428] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a: Time spent starting tablet: real 0.016s user 0.011s sys 0.000s
I20260809 15:50:48.931591 10490 tablet_bootstrap.cc:492] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4: No bootstrap required, opened a new log
I20260809 15:50:48.932495 10493 tablet_bootstrap.cc:492] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a: Bootstrap starting.
I20260809 15:50:48.932564 10490 ts_tablet_manager.cc:1397] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4: Time spent bootstrapping tablet: real 0.074s user 0.008s sys 0.004s
I20260809 15:50:48.929962 10503 raft_consensus.cc:482] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:48.934477 10503 raft_consensus.cc:504] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.938215 10490 raft_consensus.cc:348] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.938802 10490 raft_consensus.cc:374] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.939028 10490 raft_consensus.cc:729] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.939877 10490 consensus_queue.cc:260] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.942111 10495 tablet_bootstrap.cc:492] T 5ec90e88181c4e5085d12f183609cbae P 972dcf60b2944faa9d0ceec67f064e5a: No bootstrap required, opened a new log
I20260809 15:50:48.942495 10495 ts_tablet_manager.cc:1397] T 5ec90e88181c4e5085d12f183609cbae P 972dcf60b2944faa9d0ceec67f064e5a: Time spent bootstrapping tablet: real 0.028s user 0.007s sys 0.011s
I20260809 15:50:48.943945 10490 ts_tablet_manager.cc:1428] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260809 15:50:48.944105 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a" is_pre_election: true
I20260809 15:50:48.944753 10490 tablet_bootstrap.cc:492] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4: Bootstrap starting.
I20260809 15:50:48.944739 10495 raft_consensus.cc:348] T 5ec90e88181c4e5085d12f183609cbae P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.945444 10495 raft_consensus.cc:374] T 5ec90e88181c4e5085d12f183609cbae P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.945686 10495 raft_consensus.cc:729] T 5ec90e88181c4e5085d12f183609cbae P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.946252 10495 consensus_queue.cc:260] T 5ec90e88181c4e5085d12f183609cbae P 972dcf60b2944faa9d0ceec67f064e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.946808 10273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e5163456455ca4b8c5a7736031f4" is_pre_election: true
I20260809 15:50:48.947500 10273 raft_consensus.cc:2379] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 972dcf60b2944faa9d0ceec67f064e5a in current term 1: Already voted for candidate df82df23ca924767bfc57e19c09e245a in this term.
I20260809 15:50:48.948659 10495 ts_tablet_manager.cc:1428] T 5ec90e88181c4e5085d12f183609cbae P 972dcf60b2944faa9d0ceec67f064e5a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 15:50:48.948771 10401 leader_election.cc:302] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 972dcf60b2944faa9d0ceec67f064e5a; no voters: 4932e5163456455ca4b8c5a7736031f4, df82df23ca924767bfc57e19c09e245a
I20260809 15:50:48.949431 10522 raft_consensus.cc:3037] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:48.949954 10493 tablet_bootstrap.cc:654] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.950556 10490 tablet_bootstrap.cc:654] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.949436 10503 leader_election.cc:288] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197), 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561)
I20260809 15:50:48.955721 10493 tablet_bootstrap.cc:492] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a: No bootstrap required, opened a new log
I20260809 15:50:48.956197 10493 ts_tablet_manager.cc:1397] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a: Time spent bootstrapping tablet: real 0.024s user 0.007s sys 0.002s
I20260809 15:50:48.956224 10522 raft_consensus.cc:2726] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:50:48.957446 10495 tablet_bootstrap.cc:492] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a: Bootstrap starting.
I20260809 15:50:48.958837 10493 raft_consensus.cc:348] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.959595 10493 raft_consensus.cc:374] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.959880 10493 raft_consensus.cc:729] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.961063 10490 tablet_bootstrap.cc:492] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4: No bootstrap required, opened a new log
I20260809 15:50:48.960722 10493 consensus_queue.cc:260] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.961462 10490 ts_tablet_manager.cc:1397] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.007s
I20260809 15:50:48.962889 10495 tablet_bootstrap.cc:654] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.963047 10493 ts_tablet_manager.cc:1428] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a: Time spent starting tablet: real 0.006s user 0.005s sys 0.002s
I20260809 15:50:48.964236 10493 tablet_bootstrap.cc:492] T b31c1906f58b40d48decf694093d62a3 P df82df23ca924767bfc57e19c09e245a: Bootstrap starting.
I20260809 15:50:48.964023 10490 raft_consensus.cc:348] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.964692 10490 raft_consensus.cc:374] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.964941 10490 raft_consensus.cc:729] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.965680 10490 consensus_queue.cc:260] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.970479 10490 ts_tablet_manager.cc:1428] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260809 15:50:48.971282 10490 tablet_bootstrap.cc:492] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4: Bootstrap starting.
I20260809 15:50:48.971449 10493 tablet_bootstrap.cc:654] T b31c1906f58b40d48decf694093d62a3 P df82df23ca924767bfc57e19c09e245a: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.977010 10490 tablet_bootstrap.cc:654] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.978837 10495 tablet_bootstrap.cc:492] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a: No bootstrap required, opened a new log
I20260809 15:50:48.979305 10495 ts_tablet_manager.cc:1397] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a: Time spent bootstrapping tablet: real 0.022s user 0.014s sys 0.004s
I20260809 15:50:48.980677 10493 tablet_bootstrap.cc:492] T b31c1906f58b40d48decf694093d62a3 P df82df23ca924767bfc57e19c09e245a: No bootstrap required, opened a new log
I20260809 15:50:48.981086 10493 ts_tablet_manager.cc:1397] T b31c1906f58b40d48decf694093d62a3 P df82df23ca924767bfc57e19c09e245a: Time spent bootstrapping tablet: real 0.017s user 0.010s sys 0.003s
I20260809 15:50:48.970327 10155 catalog_manager.cc:5495] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3). New cstate: current_term: 1 leader_uuid: "972dcf60b2944faa9d0ceec67f064e5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: HEALTHY } } }
I20260809 15:50:48.981842 10495 raft_consensus.cc:348] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.982683 10495 raft_consensus.cc:374] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.982975 10495 raft_consensus.cc:729] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.983495 10493 raft_consensus.cc:348] T b31c1906f58b40d48decf694093d62a3 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.984130 10493 raft_consensus.cc:374] T b31c1906f58b40d48decf694093d62a3 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.983708 10495 consensus_queue.cc:260] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.984380 10493 raft_consensus.cc:729] T b31c1906f58b40d48decf694093d62a3 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.985110 10493 consensus_queue.cc:260] T b31c1906f58b40d48decf694093d62a3 P df82df23ca924767bfc57e19c09e245a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.986198 10495 ts_tablet_manager.cc:1428] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a: Time spent starting tablet: real 0.007s user 0.004s sys 0.001s
I20260809 15:50:48.987185 10493 ts_tablet_manager.cc:1428] T b31c1906f58b40d48decf694093d62a3 P df82df23ca924767bfc57e19c09e245a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 15:50:48.988013 10493 tablet_bootstrap.cc:492] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a: Bootstrap starting.
I20260809 15:50:48.988569 10490 tablet_bootstrap.cc:492] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4: No bootstrap required, opened a new log
I20260809 15:50:48.988907 10490 ts_tablet_manager.cc:1397] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4: Time spent bootstrapping tablet: real 0.018s user 0.004s sys 0.011s
I20260809 15:50:48.991060 10490 raft_consensus.cc:348] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.991691 10490 raft_consensus.cc:374] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:48.991930 10490 raft_consensus.cc:729] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Initialized, Role: FOLLOWER
I20260809 15:50:48.992606 10490 consensus_queue.cc:260] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.993192 10493 tablet_bootstrap.cc:654] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:48.992386 10156 catalog_manager.cc:5495] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a reported cstate change: term changed from 0 to 1, leader changed from <none> to df82df23ca924767bfc57e19c09e245a (127.9.84.2). New cstate: current_term: 1 leader_uuid: "df82df23ca924767bfc57e19c09e245a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:48.994415 10490 ts_tablet_manager.cc:1428] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260809 15:50:48.996438 10528 raft_consensus.cc:482] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:48.996853 10528 raft_consensus.cc:504] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:48.999243 10528 leader_election.cc:288] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:48.999965 10273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0af688bc961452abb333bd49c4c595d" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e5163456455ca4b8c5a7736031f4" is_pre_election: true
I20260809 15:50:49.000715 10273 raft_consensus.cc:2379] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate df82df23ca924767bfc57e19c09e245a in current term 1: Already voted for candidate 972dcf60b2944faa9d0ceec67f064e5a in this term.
I20260809 15:50:49.001500 10444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0af688bc961452abb333bd49c4c595d" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a" is_pre_election: true
I20260809 15:50:49.003811 10321 leader_election.cc:302] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: df82df23ca924767bfc57e19c09e245a; no voters: 4932e5163456455ca4b8c5a7736031f4, 972dcf60b2944faa9d0ceec67f064e5a
I20260809 15:50:49.004861 10528 raft_consensus.cc:3037] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:49.011680 10493 tablet_bootstrap.cc:492] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a: No bootstrap required, opened a new log
I20260809 15:50:49.011755 10528 raft_consensus.cc:2726] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:50:49.012070 10493 ts_tablet_manager.cc:1397] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a: Time spent bootstrapping tablet: real 0.024s user 0.011s sys 0.010s
I20260809 15:50:49.014443 10493 raft_consensus.cc:348] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.015002 10493 raft_consensus.cc:374] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:49.015208 10493 raft_consensus.cc:729] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Initialized, Role: FOLLOWER
I20260809 15:50:49.015671 10493 consensus_queue.cc:260] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.017098 10493 ts_tablet_manager.cc:1428] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 15:50:49.017745 10493 tablet_bootstrap.cc:492] T eb07891a9e804c51a61318118a0e8886 P df82df23ca924767bfc57e19c09e245a: Bootstrap starting.
I20260809 15:50:49.021602 10493 tablet_bootstrap.cc:654] T eb07891a9e804c51a61318118a0e8886 P df82df23ca924767bfc57e19c09e245a: Neither blocks nor log segments found. Creating new log.
I20260809 15:50:49.025275 10493 tablet_bootstrap.cc:492] T eb07891a9e804c51a61318118a0e8886 P df82df23ca924767bfc57e19c09e245a: No bootstrap required, opened a new log
I20260809 15:50:49.025655 10493 ts_tablet_manager.cc:1397] T eb07891a9e804c51a61318118a0e8886 P df82df23ca924767bfc57e19c09e245a: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 15:50:49.027201 10493 raft_consensus.cc:348] T eb07891a9e804c51a61318118a0e8886 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.027659 10493 raft_consensus.cc:374] T eb07891a9e804c51a61318118a0e8886 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:50:49.027700 10496 raft_consensus.cc:482] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:49.027925 10493 raft_consensus.cc:729] T eb07891a9e804c51a61318118a0e8886 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Initialized, Role: FOLLOWER
I20260809 15:50:49.028148 10496 raft_consensus.cc:504] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.028496 10493 consensus_queue.cc:260] T eb07891a9e804c51a61318118a0e8886 P df82df23ca924767bfc57e19c09e245a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.030012 10496 leader_election.cc:288] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:49.030268 10493 ts_tablet_manager.cc:1428] T eb07891a9e804c51a61318118a0e8886 P df82df23ca924767bfc57e19c09e245a: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20260809 15:50:49.030807 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b118cb45b2544ec68799e11793d4b481" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a" is_pre_election: true
I20260809 15:50:49.031108 10444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b118cb45b2544ec68799e11793d4b481" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a" is_pre_election: true
I20260809 15:50:49.031261 10358 raft_consensus.cc:2454] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 0.
I20260809 15:50:49.031561 10444 raft_consensus.cc:2454] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 0.
I20260809 15:50:49.032259 10238 leader_election.cc:302] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:50:49.033090 10496 raft_consensus.cc:2781] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:50:49.033417 10496 raft_consensus.cc:482] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:50:49.033751 10496 raft_consensus.cc:3037] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:49.039697 10496 raft_consensus.cc:504] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.041242 10496 leader_election.cc:288] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 1 election: Requested vote from peers df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:49.041823 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b118cb45b2544ec68799e11793d4b481" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:50:49.042443 10358 raft_consensus.cc:3037] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:49.042682 10444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b118cb45b2544ec68799e11793d4b481" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
I20260809 15:50:49.043236 10444 raft_consensus.cc:3037] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:49.046543 10358 raft_consensus.cc:2454] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 1.
I20260809 15:50:49.047171 10444 raft_consensus.cc:2454] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 1.
I20260809 15:50:49.047359 10238 leader_election.cc:302] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:50:49.048050 10496 raft_consensus.cc:2781] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:49.048884 10496 raft_consensus.cc:686] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 1 LEADER]: Becoming Leader. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Running, Role: LEADER
I20260809 15:50:49.049850 10496 consensus_queue.cc:237] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.052214 10522 raft_consensus.cc:482] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:49.052680 10522 raft_consensus.cc:504] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.054713 10522 leader_election.cc:288] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197)
I20260809 15:50:49.055413 10273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ff800c08cd44158fc25d6fd4735dc2" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e5163456455ca4b8c5a7736031f4" is_pre_election: true
I20260809 15:50:49.055967 10273 raft_consensus.cc:2454] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 0.
I20260809 15:50:49.055867 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ff800c08cd44158fc25d6fd4735dc2" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a" is_pre_election: true
I20260809 15:50:49.056560 10358 raft_consensus.cc:2454] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 0.
I20260809 15:50:49.056896 10401 leader_election.cc:302] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, 972dcf60b2944faa9d0ceec67f064e5a; no voters:
I20260809 15:50:49.056886 10155 catalog_manager.cc:5495] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "4932e5163456455ca4b8c5a7736031f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:49.057777 10522 raft_consensus.cc:2781] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:50:49.058177 10522 raft_consensus.cc:482] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:50:49.058564 10522 raft_consensus.cc:3037] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:49.064011 10522 raft_consensus.cc:504] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.065485 10522 leader_election.cc:288] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 election: Requested vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197)
I20260809 15:50:49.066248 10273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ff800c08cd44158fc25d6fd4735dc2" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
I20260809 15:50:49.066552 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ff800c08cd44158fc25d6fd4735dc2" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:50:49.066692 10273 raft_consensus.cc:3037] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:49.067054 10358 raft_consensus.cc:3037] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:49.070755 10273 raft_consensus.cc:2454] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 1.
I20260809 15:50:49.070909 10358 raft_consensus.cc:2454] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 1.
I20260809 15:50:49.071630 10401 leader_election.cc:302] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, 972dcf60b2944faa9d0ceec67f064e5a; no voters:
I20260809 15:50:49.072238 10522 raft_consensus.cc:2781] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:49.072682 10522 raft_consensus.cc:686] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Becoming Leader. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Running, Role: LEADER
I20260809 15:50:49.073402 10522 consensus_queue.cc:237] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.079504 10155 catalog_manager.cc:5495] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3). New cstate: current_term: 1 leader_uuid: "972dcf60b2944faa9d0ceec67f064e5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: HEALTHY } } }
I20260809 15:50:49.089156 10496 raft_consensus.cc:482] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:49.089542 10496 raft_consensus.cc:504] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.090988 10496 leader_election.cc:288] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:49.091619 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b31c1906f58b40d48decf694093d62a3" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a" is_pre_election: true
I20260809 15:50:49.091908 10444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b31c1906f58b40d48decf694093d62a3" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a" is_pre_election: true
I20260809 15:50:49.092094 10358 raft_consensus.cc:2454] T b31c1906f58b40d48decf694093d62a3 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 0.
I20260809 15:50:49.092442 10444 raft_consensus.cc:2454] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 0.
I20260809 15:50:49.093039 10238 leader_election.cc:302] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:50:49.093758 10496 raft_consensus.cc:2781] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:50:49.094077 10496 raft_consensus.cc:482] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:50:49.094363 10496 raft_consensus.cc:3037] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:49.098349 10496 raft_consensus.cc:504] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.099682 10496 leader_election.cc:288] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 1 election: Requested vote from peers df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:49.100471 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b31c1906f58b40d48decf694093d62a3" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:50:49.100590 10444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b31c1906f58b40d48decf694093d62a3" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
I20260809 15:50:49.100912 10358 raft_consensus.cc:3037] T b31c1906f58b40d48decf694093d62a3 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:49.101109 10444 raft_consensus.cc:3037] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:49.105365 10444 raft_consensus.cc:2454] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 1.
I20260809 15:50:49.106319 10237 leader_election.cc:302] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, 972dcf60b2944faa9d0ceec67f064e5a; no voters:
I20260809 15:50:49.106438 10358 raft_consensus.cc:2454] T b31c1906f58b40d48decf694093d62a3 P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 1.
I20260809 15:50:49.106989 10496 raft_consensus.cc:2781] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:49.107462 10496 raft_consensus.cc:686] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [term 1 LEADER]: Becoming Leader. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Running, Role: LEADER
I20260809 15:50:49.108232 10496 consensus_queue.cc:237] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.114945 10156 catalog_manager.cc:5495] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "4932e5163456455ca4b8c5a7736031f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:49.119602 10358 raft_consensus.cc:3037] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:49.125916 10522 consensus_queue.cc:1035] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:49.137229 10522 consensus_queue.cc:1035] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:49.138669 10496 raft_consensus.cc:482] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:49.139053 10496 raft_consensus.cc:504] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.140874 10496 leader_election.cc:288] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:49.141961 10522 raft_consensus.cc:482] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:50:49.142465 10522 raft_consensus.cc:504] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.142728 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ec90e88181c4e5085d12f183609cbae" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a" is_pre_election: true
I20260809 15:50:49.143452 10358 raft_consensus.cc:2454] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 0.
I20260809 15:50:49.144327 10444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ec90e88181c4e5085d12f183609cbae" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a" is_pre_election: true
I20260809 15:50:49.144999 10444 raft_consensus.cc:2454] T 5ec90e88181c4e5085d12f183609cbae P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 0.
I20260809 15:50:49.145248 10274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb07891a9e804c51a61318118a0e8886" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e5163456455ca4b8c5a7736031f4" is_pre_election: true
I20260809 15:50:49.145957 10274 raft_consensus.cc:2454] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 0.
I20260809 15:50:49.144378 10238 leader_election.cc:302] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:50:49.147486 10401 leader_election.cc:302] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, 972dcf60b2944faa9d0ceec67f064e5a; no voters:
I20260809 15:50:49.148578 10503 raft_consensus.cc:2781] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:50:49.149025 10503 raft_consensus.cc:482] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:50:49.149436 10503 raft_consensus.cc:3037] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:49.150861 10496 raft_consensus.cc:2781] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:50:49.151235 10496 raft_consensus.cc:482] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:50:49.151546 10496 raft_consensus.cc:3037] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:49.154686 10522 leader_election.cc:288] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197)
I20260809 15:50:49.155786 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb07891a9e804c51a61318118a0e8886" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a" is_pre_election: true
I20260809 15:50:49.156383 10358 raft_consensus.cc:2454] T eb07891a9e804c51a61318118a0e8886 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 0.
I20260809 15:50:49.158265 10503 raft_consensus.cc:504] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.158802 10496 raft_consensus.cc:504] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.161123 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb07891a9e804c51a61318118a0e8886" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:50:49.161396 10444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ec90e88181c4e5085d12f183609cbae" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
I20260809 15:50:49.161439 10360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ec90e88181c4e5085d12f183609cbae" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:50:49.161984 10444 raft_consensus.cc:3037] T 5ec90e88181c4e5085d12f183609cbae P 972dcf60b2944faa9d0ceec67f064e5a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:49.162123 10360 raft_consensus.cc:3037] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:49.162287 10274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb07891a9e804c51a61318118a0e8886" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
I20260809 15:50:49.162860 10274 raft_consensus.cc:3037] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:49.166206 10496 leader_election.cc:288] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 1 election: Requested vote from peers df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:49.166407 10444 raft_consensus.cc:2454] T 5ec90e88181c4e5085d12f183609cbae P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 1.
I20260809 15:50:49.167387 10237 leader_election.cc:302] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, 972dcf60b2944faa9d0ceec67f064e5a; no voters:
I20260809 15:50:49.161670 10358 raft_consensus.cc:3037] T eb07891a9e804c51a61318118a0e8886 P df82df23ca924767bfc57e19c09e245a [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:50:49.168771 10360 raft_consensus.cc:2454] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 1.
I20260809 15:50:49.169169 10274 raft_consensus.cc:2454] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 1.
I20260809 15:50:49.168799 10496 raft_consensus.cc:2781] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:49.170140 10401 leader_election.cc:302] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, 972dcf60b2944faa9d0ceec67f064e5a; no voters:
I20260809 15:50:49.170874 10496 raft_consensus.cc:686] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [term 1 LEADER]: Becoming Leader. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Running, Role: LEADER
I20260809 15:50:49.170989 10522 raft_consensus.cc:2781] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:50:49.171442 10522 raft_consensus.cc:686] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Becoming Leader. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Running, Role: LEADER
I20260809 15:50:49.171723 10496 consensus_queue.cc:237] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.172228 10522 consensus_queue.cc:237] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.174206 10503 leader_election.cc:288] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 1 election: Requested vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197)
I20260809 15:50:49.174608 10358 raft_consensus.cc:2454] T eb07891a9e804c51a61318118a0e8886 P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 1.
I20260809 15:50:49.182605 10156 catalog_manager.cc:5495] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "4932e5163456455ca4b8c5a7736031f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:49.182804 10155 catalog_manager.cc:5495] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a reported cstate change: term changed from 0 to 1, leader changed from <none> to 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3). New cstate: current_term: 1 leader_uuid: "972dcf60b2944faa9d0ceec67f064e5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: HEALTHY } } }
I20260809 15:50:49.283790 10528 consensus_queue.cc:1035] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [LEADER]: Connected to new peer: Peer: permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:49.294021 10534 consensus_queue.cc:1035] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [LEADER]: Connected to new peer: Peer: permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:49.295789 10566 consensus_queue.cc:1035] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260809 15:50:49.299007 10521 consensus_queue.cc:1035] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:49.310997 10528 consensus_queue.cc:1035] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:49.313752 10503 consensus_queue.cc:1035] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:49.349836 10358 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01"
dest_uuid: "df82df23ca924767bfc57e19c09e245a"
mode: GRACEFUL
new_leader_uuid: "4932e5163456455ca4b8c5a7736031f4"
from {username='slave'} at 127.0.0.1:41384
I20260809 15:50:49.350718 10358 raft_consensus.cc:595] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 1 LEADER]: Received request to transfer leadership to TS 4932e5163456455ca4b8c5a7736031f4
I20260809 15:50:49.364640 10444 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "eb07891a9e804c51a61318118a0e8886"
dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
mode: GRACEFUL
new_leader_uuid: "4932e5163456455ca4b8c5a7736031f4"
from {username='slave'} at 127.0.0.1:39730
I20260809 15:50:49.365114 10444 raft_consensus.cc:595] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Received request to transfer leadership to TS 4932e5163456455ca4b8c5a7736031f4
I20260809 15:50:49.371634 10444 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b5cc2389e9a74343b1e06708819e63c2"
dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
mode: GRACEFUL
new_leader_uuid: "4932e5163456455ca4b8c5a7736031f4"
from {username='slave'} at 127.0.0.1:39730
I20260809 15:50:49.372061 10444 raft_consensus.cc:595] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Received request to transfer leadership to TS 4932e5163456455ca4b8c5a7736031f4
I20260809 15:50:49.376422 10444 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b0af688bc961452abb333bd49c4c595d"
dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
mode: GRACEFUL
new_leader_uuid: "4932e5163456455ca4b8c5a7736031f4"
from {username='slave'} at 127.0.0.1:39730
I20260809 15:50:49.376811 10444 raft_consensus.cc:595] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Received request to transfer leadership to TS 4932e5163456455ca4b8c5a7736031f4
I20260809 15:50:49.380662 10444 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "00c2886e82e647e6986c2ace7f0fb5d2"
dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
mode: GRACEFUL
new_leader_uuid: "df82df23ca924767bfc57e19c09e245a"
from {username='slave'} at 127.0.0.1:39730
I20260809 15:50:49.381049 10444 raft_consensus.cc:595] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Received request to transfer leadership to TS df82df23ca924767bfc57e19c09e245a
I20260809 15:50:49.385048 10444 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a4ff800c08cd44158fc25d6fd4735dc2"
dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
mode: GRACEFUL
new_leader_uuid: "df82df23ca924767bfc57e19c09e245a"
from {username='slave'} at 127.0.0.1:39730
I20260809 15:50:49.385416 10444 raft_consensus.cc:595] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Received request to transfer leadership to TS df82df23ca924767bfc57e19c09e245a
I20260809 15:50:49.396596 10503 consensus_queue.cc:1035] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:49.398001 10503 consensus_queue.cc:1035] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:49.399765 10273 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b118cb45b2544ec68799e11793d4b481"
dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
mode: GRACEFUL
new_leader_uuid: "df82df23ca924767bfc57e19c09e245a"
from {username='slave'} at 127.0.0.1:38940
I20260809 15:50:49.400907 10273 raft_consensus.cc:595] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 1 LEADER]: Received request to transfer leadership to TS df82df23ca924767bfc57e19c09e245a
I20260809 15:50:49.407616 10522 raft_consensus.cc:979] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a: : Instructing follower 4932e5163456455ca4b8c5a7736031f4 to start an election
I20260809 15:50:49.407991 10522 raft_consensus.cc:1067] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Signalling peer 4932e5163456455ca4b8c5a7736031f4 to start an election
I20260809 15:50:49.409467 10273 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b0af688bc961452abb333bd49c4c595d"
dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
from {username='slave'} at 127.0.0.1:38918
I20260809 15:50:49.409981 10273 raft_consensus.cc:482] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:49.410367 10273 raft_consensus.cc:3037] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.417495 10273 raft_consensus.cc:504] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.417949 10275 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b31c1906f58b40d48decf694093d62a3"
dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
mode: GRACEFUL
new_leader_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
from {username='slave'} at 127.0.0.1:38940
I20260809 15:50:49.418443 10274 raft_consensus.cc:1226] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Rejecting Update request from peer 972dcf60b2944faa9d0ceec67f064e5a for earlier term 1. Current term is 2. Ops: []
I20260809 15:50:49.418601 10275 raft_consensus.cc:595] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [term 1 LEADER]: Received request to transfer leadership to TS 972dcf60b2944faa9d0ceec67f064e5a
I20260809 15:50:49.422217 10522 consensus_queue.cc:1046] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:49.422669 10273 leader_election.cc:288] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 2 election: Requested vote from peers df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:49.422555 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0af688bc961452abb333bd49c4c595d" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:50:49.423185 10444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0af688bc961452abb333bd49c4c595d" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
I20260809 15:50:49.423355 10522 raft_consensus.cc:3032] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:50:49.423405 10358 raft_consensus.cc:3037] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.423732 10522 raft_consensus.cc:729] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Running, Role: LEADER
I20260809 15:50:49.424568 10522 consensus_queue.cc:260] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.425581 10522 raft_consensus.cc:3037] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.431039 10358 raft_consensus.cc:2454] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 2.
I20260809 15:50:49.431946 10238 leader_election.cc:302] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:50:49.432679 10496 raft_consensus.cc:2781] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:50:49.433065 10496 raft_consensus.cc:686] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [term 2 LEADER]: Becoming Leader. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Running, Role: LEADER
I20260809 15:50:49.433804 10496 consensus_queue.cc:237] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.436259 10444 raft_consensus.cc:2454] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 2.
I20260809 15:50:49.440204 10155 catalog_manager.cc:5495] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 reported cstate change: term changed from 1 to 2, leader changed from 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3) to 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1). New cstate: current_term: 2 leader_uuid: "4932e5163456455ca4b8c5a7736031f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:49.462965 10496 consensus_queue.cc:1035] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:49.469123 10554 raft_consensus.cc:979] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4: : Instructing follower df82df23ca924767bfc57e19c09e245a to start an election
I20260809 15:50:49.469544 10496 raft_consensus.cc:1067] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 1 LEADER]: Signalling peer df82df23ca924767bfc57e19c09e245a to start an election
I20260809 15:50:49.473719 10360 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b118cb45b2544ec68799e11793d4b481"
dest_uuid: "df82df23ca924767bfc57e19c09e245a"
from {username='slave'} at 127.0.0.1:41376
I20260809 15:50:49.474360 10360 raft_consensus.cc:482] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:49.474704 10360 raft_consensus.cc:3037] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.476768 10496 consensus_queue.cc:1035] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:49.481596 10360 raft_consensus.cc:504] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.485550 10360 leader_election.cc:288] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 2 election: Requested vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:49.486259 10444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b118cb45b2544ec68799e11793d4b481" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
I20260809 15:50:49.486943 10444 raft_consensus.cc:3037] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.488389 10274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b118cb45b2544ec68799e11793d4b481" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
I20260809 15:50:49.488988 10274 raft_consensus.cc:3032] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:50:49.489315 10274 raft_consensus.cc:729] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Running, Role: LEADER
I20260809 15:50:49.490228 10274 consensus_queue.cc:260] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.491612 10274 raft_consensus.cc:3037] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.494553 10444 raft_consensus.cc:2454] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df82df23ca924767bfc57e19c09e245a in term 2.
I20260809 15:50:49.495761 10321 leader_election.cc:302] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 972dcf60b2944faa9d0ceec67f064e5a, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:50:49.496587 10528 raft_consensus.cc:2781] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:50:49.497035 10528 raft_consensus.cc:686] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 2 LEADER]: Becoming Leader. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Running, Role: LEADER
I20260809 15:50:49.497790 10528 consensus_queue.cc:237] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.498310 10274 raft_consensus.cc:2454] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df82df23ca924767bfc57e19c09e245a in term 2.
I20260809 15:50:49.501210 10522 consensus_queue.cc:1035] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:49.505163 10155 catalog_manager.cc:5495] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a reported cstate change: term changed from 1 to 2, leader changed from 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1) to df82df23ca924767bfc57e19c09e245a (127.9.84.2). New cstate: current_term: 2 leader_uuid: "df82df23ca924767bfc57e19c09e245a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:49.511933 10503 consensus_queue.cc:1035] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:49.520690 10521 raft_consensus.cc:979] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a: : Instructing follower df82df23ca924767bfc57e19c09e245a to start an election
I20260809 15:50:49.521116 10503 raft_consensus.cc:1067] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Signalling peer df82df23ca924767bfc57e19c09e245a to start an election
I20260809 15:50:49.522532 10360 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a4ff800c08cd44158fc25d6fd4735dc2"
dest_uuid: "df82df23ca924767bfc57e19c09e245a"
from {username='slave'} at 127.0.0.1:41374
I20260809 15:50:49.522972 10360 raft_consensus.cc:482] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:49.523283 10360 raft_consensus.cc:3037] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.528420 10360 raft_consensus.cc:504] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.530115 10360 leader_election.cc:288] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 2 election: Requested vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:49.531550 10444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ff800c08cd44158fc25d6fd4735dc2" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
I20260809 15:50:49.531507 10273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ff800c08cd44158fc25d6fd4735dc2" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
I20260809 15:50:49.532203 10444 raft_consensus.cc:3032] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:50:49.532301 10273 raft_consensus.cc:3037] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.532563 10444 raft_consensus.cc:729] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Running, Role: LEADER
I20260809 15:50:49.533326 10444 consensus_queue.cc:260] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.534533 10444 raft_consensus.cc:3037] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.538533 10496 consensus_queue.cc:1035] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:49.538635 10273 raft_consensus.cc:2454] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df82df23ca924767bfc57e19c09e245a in term 2.
I20260809 15:50:49.539750 10316 leader_election.cc:302] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:50:49.540374 10528 raft_consensus.cc:2781] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:50:49.540637 10444 raft_consensus.cc:2454] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df82df23ca924767bfc57e19c09e245a in term 2.
I20260809 15:50:49.540901 10528 raft_consensus.cc:686] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [term 2 LEADER]: Becoming Leader. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Running, Role: LEADER
I20260809 15:50:49.542045 10528 consensus_queue.cc:237] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.549798 10158 catalog_manager.cc:5495] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a reported cstate change: term changed from 1 to 2, leader changed from 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3) to df82df23ca924767bfc57e19c09e245a (127.9.84.2). New cstate: current_term: 2 leader_uuid: "df82df23ca924767bfc57e19c09e245a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:49.550575 10554 consensus_queue.cc:1035] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:49.558269 10581 raft_consensus.cc:979] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4: : Instructing follower 972dcf60b2944faa9d0ceec67f064e5a to start an election
I20260809 15:50:49.558728 10581 raft_consensus.cc:1067] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [term 1 LEADER]: Signalling peer 972dcf60b2944faa9d0ceec67f064e5a to start an election
I20260809 15:50:49.560317 10444 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b31c1906f58b40d48decf694093d62a3"
dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
from {username='slave'} at 127.0.0.1:39714
I20260809 15:50:49.560842 10444 raft_consensus.cc:482] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:49.561228 10444 raft_consensus.cc:3037] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.566277 10444 raft_consensus.cc:504] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.567888 10503 raft_consensus.cc:979] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a: : Instructing follower df82df23ca924767bfc57e19c09e245a to start an election
I20260809 15:50:49.567956 10444 leader_election.cc:288] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 2 election: Requested vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197)
I20260809 15:50:49.568367 10503 raft_consensus.cc:1067] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Signalling peer df82df23ca924767bfc57e19c09e245a to start an election
I20260809 15:50:49.568598 10273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b31c1906f58b40d48decf694093d62a3" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
I20260809 15:50:49.569008 10360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b31c1906f58b40d48decf694093d62a3" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:50:49.569182 10273 raft_consensus.cc:3032] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:50:49.569581 10360 raft_consensus.cc:3037] T b31c1906f58b40d48decf694093d62a3 P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.569581 10273 raft_consensus.cc:729] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Running, Role: LEADER
I20260809 15:50:49.569975 10358 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00c2886e82e647e6986c2ace7f0fb5d2"
dest_uuid: "df82df23ca924767bfc57e19c09e245a"
from {username='slave'} at 127.0.0.1:41374
I20260809 15:50:49.570533 10358 raft_consensus.cc:482] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:49.570889 10358 raft_consensus.cc:3037] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.571529 10273 consensus_queue.cc:260] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.572824 10273 raft_consensus.cc:3037] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.575946 10360 raft_consensus.cc:2454] T b31c1906f58b40d48decf694093d62a3 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 2.
I20260809 15:50:49.576860 10406 leader_election.cc:302] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 972dcf60b2944faa9d0ceec67f064e5a, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:50:49.577234 10358 raft_consensus.cc:504] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.577663 10521 raft_consensus.cc:2781] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:50:49.577932 10273 raft_consensus.cc:2454] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 2.
I20260809 15:50:49.578194 10521 raft_consensus.cc:686] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 LEADER]: Becoming Leader. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Running, Role: LEADER
I20260809 15:50:49.579211 10358 leader_election.cc:288] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 2 election: Requested vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:49.579012 10521 consensus_queue.cc:237] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.579852 10273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00c2886e82e647e6986c2ace7f0fb5d2" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
I20260809 15:50:49.580375 10273 raft_consensus.cc:3037] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.580541 10444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00c2886e82e647e6986c2ace7f0fb5d2" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
I20260809 15:50:49.581136 10444 raft_consensus.cc:3032] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:50:49.581435 10444 raft_consensus.cc:729] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Running, Role: LEADER
I20260809 15:50:49.582185 10444 consensus_queue.cc:260] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.583263 10444 raft_consensus.cc:3037] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.585834 10273 raft_consensus.cc:2454] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df82df23ca924767bfc57e19c09e245a in term 2.
I20260809 15:50:49.586869 10316 leader_election.cc:302] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:50:49.586619 10155 catalog_manager.cc:5495] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a reported cstate change: term changed from 1 to 2, leader changed from 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1) to 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3). New cstate: current_term: 2 leader_uuid: "972dcf60b2944faa9d0ceec67f064e5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: HEALTHY } } }
I20260809 15:50:49.587628 10528 raft_consensus.cc:2781] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:50:49.588171 10528 raft_consensus.cc:686] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 2 LEADER]: Becoming Leader. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Running, Role: LEADER
I20260809 15:50:49.588862 10528 consensus_queue.cc:237] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.587628 10444 raft_consensus.cc:2454] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df82df23ca924767bfc57e19c09e245a in term 2.
I20260809 15:50:49.595112 10156 catalog_manager.cc:5495] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a reported cstate change: term changed from 1 to 2, leader changed from 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3) to df82df23ca924767bfc57e19c09e245a (127.9.84.2). New cstate: current_term: 2 leader_uuid: "df82df23ca924767bfc57e19c09e245a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:49.643899 10554 consensus_queue.cc:1035] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:49.653090 10581 consensus_queue.cc:1035] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:49.666700 10521 consensus_queue.cc:1035] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:49.672866 10503 raft_consensus.cc:979] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a: : Instructing follower 4932e5163456455ca4b8c5a7736031f4 to start an election
I20260809 15:50:49.673844 10522 raft_consensus.cc:1067] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Signalling peer 4932e5163456455ca4b8c5a7736031f4 to start an election
I20260809 15:50:49.676107 10274 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "eb07891a9e804c51a61318118a0e8886"
dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
from {username='slave'} at 127.0.0.1:38918
I20260809 15:50:49.676642 10274 raft_consensus.cc:482] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:49.676997 10274 raft_consensus.cc:3037] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.681311 10503 consensus_queue.cc:1035] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:50:49.682196 10274 raft_consensus.cc:504] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.684980 10360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb07891a9e804c51a61318118a0e8886" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:50:49.685068 10274 leader_election.cc:288] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 2 election: Requested vote from peers df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:49.686192 10444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb07891a9e804c51a61318118a0e8886" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
I20260809 15:50:49.686812 10444 raft_consensus.cc:3032] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:50:49.687108 10444 raft_consensus.cc:729] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Running, Role: LEADER
I20260809 15:50:49.687909 10444 consensus_queue.cc:260] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.689831 10444 raft_consensus.cc:3037] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.695214 10444 raft_consensus.cc:2454] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 2.
I20260809 15:50:49.696071 10237 leader_election.cc:302] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, 972dcf60b2944faa9d0ceec67f064e5a; no voters: df82df23ca924767bfc57e19c09e245a
I20260809 15:50:49.696712 10581 raft_consensus.cc:2781] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:50:49.697180 10581 raft_consensus.cc:686] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [term 2 LEADER]: Becoming Leader. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Running, Role: LEADER
I20260809 15:50:49.697935 10581 consensus_queue.cc:237] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.704308 10155 catalog_manager.cc:5495] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 reported cstate change: term changed from 1 to 2, leader changed from 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3) to 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1). New cstate: current_term: 2 leader_uuid: "4932e5163456455ca4b8c5a7736031f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:49.832113 10444 raft_consensus.cc:1261] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Refusing update from remote peer 4932e5163456455ca4b8c5a7736031f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:49.833284 10581 consensus_queue.cc:1035] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:49.841261 10360 raft_consensus.cc:1261] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Refusing update from remote peer 4932e5163456455ca4b8c5a7736031f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:49.842809 10496 consensus_queue.cc:1035] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:49.886579 10274 raft_consensus.cc:1261] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Refusing update from remote peer df82df23ca924767bfc57e19c09e245a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:49.887732 10528 consensus_queue.cc:1035] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:49.894282 10444 raft_consensus.cc:1261] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Refusing update from remote peer df82df23ca924767bfc57e19c09e245a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:49.895646 10566 consensus_queue.cc:1035] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [LEADER]: Connected to new peer: Peer: permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:49.909502 10521 raft_consensus.cc:979] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a: : Instructing follower 4932e5163456455ca4b8c5a7736031f4 to start an election
I20260809 15:50:49.909917 10503 raft_consensus.cc:1067] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Signalling peer 4932e5163456455ca4b8c5a7736031f4 to start an election
I20260809 15:50:49.911649 10274 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b5cc2389e9a74343b1e06708819e63c2"
dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
from {username='slave'} at 127.0.0.1:38918
I20260809 15:50:49.912075 10274 raft_consensus.cc:482] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:49.912441 10274 raft_consensus.cc:3037] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.917197 10528 raft_consensus.cc:979] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a: : Instructing follower 4932e5163456455ca4b8c5a7736031f4 to start an election
I20260809 15:50:49.917646 10566 raft_consensus.cc:1067] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 1 LEADER]: Signalling peer 4932e5163456455ca4b8c5a7736031f4 to start an election
I20260809 15:50:49.917991 10274 raft_consensus.cc:504] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.919227 10273 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01"
dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
from {username='slave'} at 127.0.0.1:38926
I20260809 15:50:49.919804 10273 raft_consensus.cc:482] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:49.920127 10273 raft_consensus.cc:3037] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.920272 10274 leader_election.cc:288] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 2 election: Requested vote from peers df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:49.920820 10360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5cc2389e9a74343b1e06708819e63c2" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:50:49.921384 10360 raft_consensus.cc:3037] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.920843 10444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5cc2389e9a74343b1e06708819e63c2" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
I20260809 15:50:49.926539 10444 raft_consensus.cc:3032] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:50:49.926980 10444 raft_consensus.cc:729] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Running, Role: LEADER
I20260809 15:50:49.926527 10273 raft_consensus.cc:504] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.927663 10360 raft_consensus.cc:2454] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 2.
I20260809 15:50:49.927695 10444 consensus_queue.cc:260] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.928608 10273 leader_election.cc:288] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 2 election: Requested vote from peers df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:50:49.928865 10444 raft_consensus.cc:3037] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.929354 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:50:49.929852 10358 raft_consensus.cc:3032] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:50:49.930147 10358 raft_consensus.cc:729] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 1 LEADER]: Becoming Follower/Learner. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Running, Role: LEADER
I20260809 15:50:49.930722 10358 consensus_queue.cc:260] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.931636 10358 raft_consensus.cc:3037] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.933701 10444 raft_consensus.cc:2454] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 2.
I20260809 15:50:49.934477 10445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
I20260809 15:50:49.934728 10237 leader_election.cc:302] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, 972dcf60b2944faa9d0ceec67f064e5a; no voters:
I20260809 15:50:49.935076 10445 raft_consensus.cc:3037] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:49.935540 10581 raft_consensus.cc:2781] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:50:49.936044 10581 raft_consensus.cc:686] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 2 LEADER]: Becoming Leader. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Running, Role: LEADER
I20260809 15:50:49.936546 10358 raft_consensus.cc:2454] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 2.
I20260809 15:50:49.936832 10581 consensus_queue.cc:237] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.937395 10238 leader_election.cc:302] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:50:49.937989 10496 raft_consensus.cc:2781] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:50:49.938573 10496 raft_consensus.cc:686] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 2 LEADER]: Becoming Leader. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Running, Role: LEADER
I20260809 15:50:49.939329 10496 consensus_queue.cc:237] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:49.943130 10445 raft_consensus.cc:2454] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 2.
I20260809 15:50:49.942939 10156 catalog_manager.cc:5495] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 reported cstate change: term changed from 1 to 2, leader changed from 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3) to 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1). New cstate: current_term: 2 leader_uuid: "4932e5163456455ca4b8c5a7736031f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:49.953019 10156 catalog_manager.cc:5495] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 reported cstate change: term changed from 1 to 2, leader changed from df82df23ca924767bfc57e19c09e245a (127.9.84.2) to 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1). New cstate: current_term: 2 leader_uuid: "4932e5163456455ca4b8c5a7736031f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: UNKNOWN } } }
I20260809 15:50:50.013412 10445 raft_consensus.cc:1261] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Refusing update from remote peer df82df23ca924767bfc57e19c09e245a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:50.014530 10566 consensus_queue.cc:1035] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [LEADER]: Connected to new peer: Peer: permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:50.024230 10273 raft_consensus.cc:1261] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Refusing update from remote peer df82df23ca924767bfc57e19c09e245a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:50.025585 10566 consensus_queue.cc:1035] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:50.038955 10445 raft_consensus.cc:1261] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Refusing update from remote peer df82df23ca924767bfc57e19c09e245a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:50.039989 10566 consensus_queue.cc:1035] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [LEADER]: Connected to new peer: Peer: permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:50.048090 10273 raft_consensus.cc:1261] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Refusing update from remote peer df82df23ca924767bfc57e19c09e245a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:50.049284 10566 consensus_queue.cc:1035] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:50:50.110769 10273 raft_consensus.cc:1261] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Refusing update from remote peer 972dcf60b2944faa9d0ceec67f064e5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:50.111970 10503 consensus_queue.cc:1035] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:50.119688 10360 raft_consensus.cc:1261] T b31c1906f58b40d48decf694093d62a3 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Refusing update from remote peer 972dcf60b2944faa9d0ceec67f064e5a: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:50.121222 10522 consensus_queue.cc:1035] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:50.141095 10445 raft_consensus.cc:1261] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Refusing update from remote peer 4932e5163456455ca4b8c5a7736031f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:50.142194 10496 consensus_queue.cc:1035] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:50.148267 10360 raft_consensus.cc:3037] T eb07891a9e804c51a61318118a0e8886 P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:50:50.154925 10360 raft_consensus.cc:1261] T eb07891a9e804c51a61318118a0e8886 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Refusing update from remote peer 4932e5163456455ca4b8c5a7736031f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:50.155982 10496 consensus_queue.cc:1035] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:50.336331 10445 raft_consensus.cc:1261] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Refusing update from remote peer 4932e5163456455ca4b8c5a7736031f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:50.337527 10496 consensus_queue.cc:1035] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:50.345381 10360 raft_consensus.cc:1261] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Refusing update from remote peer 4932e5163456455ca4b8c5a7736031f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:50.346792 10496 consensus_queue.cc:1035] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:50.448657 10360 raft_consensus.cc:1261] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Refusing update from remote peer 4932e5163456455ca4b8c5a7736031f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:50.450824 10581 consensus_queue.cc:1035] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:50.451790 10445 raft_consensus.cc:1261] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Refusing update from remote peer 4932e5163456455ca4b8c5a7736031f4: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:50:50.453020 10581 consensus_queue.cc:1035] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:50:52.447403 9552 auto_leader_rebalancer-test.cc:339] The leader distribution is
I20260809 15:50:52.448014 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:52.464524 10273 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01"
dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
mode: GRACEFUL
new_leader_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
from {username='slave'} at 127.0.0.1:38952
I20260809 15:50:52.464962 10273 raft_consensus.cc:595] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 2 LEADER]: Received request to transfer leadership to TS 972dcf60b2944faa9d0ceec67f064e5a
I20260809 15:50:52.467126 10273 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b5cc2389e9a74343b1e06708819e63c2"
dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
mode: GRACEFUL
new_leader_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
from {username='slave'} at 127.0.0.1:38952
I20260809 15:50:52.467662 10273 raft_consensus.cc:595] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 2 LEADER]: Received request to transfer leadership to TS 972dcf60b2944faa9d0ceec67f064e5a
I20260809 15:50:52.468684 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260809 15:50:52.469216 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:52.471230 10554 raft_consensus.cc:979] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4: : Instructing follower 972dcf60b2944faa9d0ceec67f064e5a to start an election
I20260809 15:50:52.471659 10602 raft_consensus.cc:1067] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 2 LEADER]: Signalling peer 972dcf60b2944faa9d0ceec67f064e5a to start an election
I20260809 15:50:52.472846 10445 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b5cc2389e9a74343b1e06708819e63c2"
dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
from {username='slave'} at 127.0.0.1:39714
I20260809 15:50:52.473266 10445 raft_consensus.cc:482] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:52.473562 10445 raft_consensus.cc:3037] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:52.477756 10445 raft_consensus.cc:504] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:52.479295 10445 leader_election.cc:288] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 3 election: Requested vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197)
I20260809 15:50:52.480134 10273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5cc2389e9a74343b1e06708819e63c2" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
I20260809 15:50:52.480355 10360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5cc2389e9a74343b1e06708819e63c2" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:50:52.480676 10273 raft_consensus.cc:3032] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:50:52.480919 10360 raft_consensus.cc:3037] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:52.481002 10273 raft_consensus.cc:729] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Running, Role: LEADER
I20260809 15:50:52.481928 10273 consensus_queue.cc:260] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:52.483333 10273 raft_consensus.cc:3037] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:52.486495 10360 raft_consensus.cc:2454] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 3.
I20260809 15:50:52.487555 10406 leader_election.cc:302] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 972dcf60b2944faa9d0ceec67f064e5a, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:50:52.488302 10521 raft_consensus.cc:2781] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:50:52.489080 10273 raft_consensus.cc:2454] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 3.
I20260809 15:50:52.491439 10521 raft_consensus.cc:686] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 LEADER]: Becoming Leader. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Running, Role: LEADER
I20260809 15:50:52.492337 10521 consensus_queue.cc:237] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:52.499107 10158 catalog_manager.cc:5495] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a reported cstate change: term changed from 2 to 3, leader changed from 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1) to 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3). New cstate: current_term: 3 leader_uuid: "972dcf60b2944faa9d0ceec67f064e5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: HEALTHY } } }
I20260809 15:50:52.748836 10554 raft_consensus.cc:979] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4: : Instructing follower 972dcf60b2944faa9d0ceec67f064e5a to start an election
I20260809 15:50:52.749276 10602 raft_consensus.cc:1067] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 2 LEADER]: Signalling peer 972dcf60b2944faa9d0ceec67f064e5a to start an election
I20260809 15:50:52.750552 10445 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01"
dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
from {username='slave'} at 127.0.0.1:39714
I20260809 15:50:52.750998 10445 raft_consensus.cc:482] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:50:52.751255 10445 raft_consensus.cc:3037] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:52.755600 10445 raft_consensus.cc:504] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:52.757045 10445 leader_election.cc:288] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 3 election: Requested vote from peers df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197), 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561)
I20260809 15:50:52.757906 10360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:50:52.758148 10273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
I20260809 15:50:52.758466 10360 raft_consensus.cc:3037] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:52.758664 10273 raft_consensus.cc:3032] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:50:52.758975 10273 raft_consensus.cc:729] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Running, Role: LEADER
I20260809 15:50:52.759593 10273 consensus_queue.cc:260] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:52.760442 10273 raft_consensus.cc:3037] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:50:52.763125 10360 raft_consensus.cc:2454] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 3.
I20260809 15:50:52.764024 10406 leader_election.cc:302] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 972dcf60b2944faa9d0ceec67f064e5a, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:50:52.764673 10521 raft_consensus.cc:2781] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:50:52.764731 10273 raft_consensus.cc:2454] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 3.
I20260809 15:50:52.765136 10521 raft_consensus.cc:686] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 LEADER]: Becoming Leader. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Running, Role: LEADER
I20260809 15:50:52.765851 10521 consensus_queue.cc:237] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:50:52.772032 10158 catalog_manager.cc:5495] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a reported cstate change: term changed from 2 to 3, leader changed from 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1) to 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3). New cstate: current_term: 3 leader_uuid: "972dcf60b2944faa9d0ceec67f064e5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: HEALTHY } } }
I20260809 15:50:52.907600 10360 raft_consensus.cc:1261] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [term 3 FOLLOWER]: Refusing update from remote peer 972dcf60b2944faa9d0ceec67f064e5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:50:52.908667 10521 consensus_queue.cc:1035] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:50:52.924144 10273 raft_consensus.cc:1261] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 3 FOLLOWER]: Refusing update from remote peer 972dcf60b2944faa9d0ceec67f064e5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:50:52.925441 10604 consensus_queue.cc:1035] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:50:53.155277 10360 raft_consensus.cc:1261] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 3 FOLLOWER]: Refusing update from remote peer 972dcf60b2944faa9d0ceec67f064e5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:50:53.156412 10604 consensus_queue.cc:1035] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:50:53.164304 10274 raft_consensus.cc:1261] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 3 FOLLOWER]: Refusing update from remote peer 972dcf60b2944faa9d0ceec67f064e5a: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:50:53.165714 10603 consensus_queue.cc:1035] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:50:53.470058 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:53.475869 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:53.476248 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:54.476963 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:54.483215 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:54.483642 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:55.484556 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:55.489694 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:55.490113 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:56.490860 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:56.496395 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:56.496917 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:57.498150 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:57.506291 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:57.506780 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:58.507668 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:58.512645 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:58.513032 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:50:59.513849 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:50:59.521783 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:50:59.522359 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:51:00.523197 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:00.529352 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:51:00.529784 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:51:01.530726 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:01.536269 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:51:01.536650 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:51:02.564540 9552 auto_leader_rebalancer-test.cc:358] The leader distribution is
I20260809 15:51:02.568884 10444 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01"
dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
mode: GRACEFUL
new_leader_uuid: "4932e5163456455ca4b8c5a7736031f4"
from {username='slave'} at 127.0.0.1:39730
I20260809 15:51:02.569559 10444 raft_consensus.cc:595] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 LEADER]: Received request to transfer leadership to TS 4932e5163456455ca4b8c5a7736031f4
I20260809 15:51:02.578459 10444 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b5cc2389e9a74343b1e06708819e63c2"
dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
mode: GRACEFUL
new_leader_uuid: "4932e5163456455ca4b8c5a7736031f4"
from {username='slave'} at 127.0.0.1:39730
I20260809 15:51:02.578997 10444 raft_consensus.cc:595] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 LEADER]: Received request to transfer leadership to TS 4932e5163456455ca4b8c5a7736031f4
I20260809 15:51:02.585570 10360 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "00c2886e82e647e6986c2ace7f0fb5d2"
dest_uuid: "df82df23ca924767bfc57e19c09e245a"
mode: GRACEFUL
new_leader_uuid: "4932e5163456455ca4b8c5a7736031f4"
from {username='slave'} at 127.0.0.1:41384
I20260809 15:51:02.586019 10360 raft_consensus.cc:595] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 2 LEADER]: Received request to transfer leadership to TS 4932e5163456455ca4b8c5a7736031f4
I20260809 15:51:02.589968 10360 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "a4ff800c08cd44158fc25d6fd4735dc2"
dest_uuid: "df82df23ca924767bfc57e19c09e245a"
mode: GRACEFUL
new_leader_uuid: "4932e5163456455ca4b8c5a7736031f4"
from {username='slave'} at 127.0.0.1:41384
I20260809 15:51:02.590418 10360 raft_consensus.cc:595] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [term 2 LEADER]: Received request to transfer leadership to TS 4932e5163456455ca4b8c5a7736031f4
I20260809 15:51:02.594316 10360 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b118cb45b2544ec68799e11793d4b481"
dest_uuid: "df82df23ca924767bfc57e19c09e245a"
mode: GRACEFUL
new_leader_uuid: "4932e5163456455ca4b8c5a7736031f4"
from {username='slave'} at 127.0.0.1:41384
I20260809 15:51:02.594733 10360 raft_consensus.cc:595] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 2 LEADER]: Received request to transfer leadership to TS 4932e5163456455ca4b8c5a7736031f4
I20260809 15:51:02.867811 10634 raft_consensus.cc:979] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a: : Instructing follower 4932e5163456455ca4b8c5a7736031f4 to start an election
I20260809 15:51:02.868535 10633 raft_consensus.cc:1067] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [term 2 LEADER]: Signalling peer 4932e5163456455ca4b8c5a7736031f4 to start an election
I20260809 15:51:02.870371 10634 raft_consensus.cc:979] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a: : Instructing follower 4932e5163456455ca4b8c5a7736031f4 to start an election
I20260809 15:51:02.870186 10274 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "a4ff800c08cd44158fc25d6fd4735dc2"
dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
from {username='slave'} at 127.0.0.1:38926
I20260809 15:51:02.870832 10633 raft_consensus.cc:1067] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 2 LEADER]: Signalling peer 4932e5163456455ca4b8c5a7736031f4 to start an election
I20260809 15:51:02.870862 10274 raft_consensus.cc:482] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:02.871407 10274 raft_consensus.cc:3037] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:51:02.872669 10273 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b118cb45b2544ec68799e11793d4b481"
dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
from {username='slave'} at 127.0.0.1:38926
I20260809 15:51:02.873184 10273 raft_consensus.cc:482] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:02.873543 10273 raft_consensus.cc:3037] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:51:02.879689 10274 raft_consensus.cc:504] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:02.880785 10273 raft_consensus.cc:504] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:02.882622 10273 leader_election.cc:288] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 3 election: Requested vote from peers df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:51:02.883880 10360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ff800c08cd44158fc25d6fd4735dc2" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:51:02.884227 10445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b118cb45b2544ec68799e11793d4b481" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
I20260809 15:51:02.884080 10358 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b118cb45b2544ec68799e11793d4b481" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:51:02.884615 10360 raft_consensus.cc:3032] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:51:02.884863 10358 raft_consensus.cc:3032] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:51:02.884838 10445 raft_consensus.cc:3037] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:51:02.885305 10358 raft_consensus.cc:729] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Running, Role: LEADER
I20260809 15:51:02.885197 10444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a4ff800c08cd44158fc25d6fd4735dc2" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
I20260809 15:51:02.884979 10360 raft_consensus.cc:729] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Running, Role: LEADER
I20260809 15:51:02.885931 10444 raft_consensus.cc:3037] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:51:02.886292 10358 consensus_queue.cc:260] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:02.886806 10360 consensus_queue.cc:260] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:02.887923 10358 raft_consensus.cc:3037] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:51:02.888010 10360 raft_consensus.cc:3037] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:51:02.894425 10445 raft_consensus.cc:2454] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 3.
I20260809 15:51:02.894694 10360 raft_consensus.cc:2454] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 3.
I20260809 15:51:02.895092 10358 raft_consensus.cc:2454] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 3.
I20260809 15:51:02.895356 10444 raft_consensus.cc:2454] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 3.
I20260809 15:51:02.895910 10237 leader_election.cc:302] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, 972dcf60b2944faa9d0ceec67f064e5a; no voters:
I20260809 15:51:02.896035 10238 leader_election.cc:302] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:51:02.896243 10274 leader_election.cc:288] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 3 election: Requested vote from peers df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:51:02.896900 10630 raft_consensus.cc:2781] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:51:02.897917 10635 raft_consensus.cc:2781] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:51:02.898286 10630 raft_consensus.cc:686] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 3 LEADER]: Becoming Leader. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Running, Role: LEADER
I20260809 15:51:02.899456 10630 consensus_queue.cc:237] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:02.901343 10635 raft_consensus.cc:686] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [term 3 LEADER]: Becoming Leader. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Running, Role: LEADER
I20260809 15:51:02.902338 10635 consensus_queue.cc:237] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:02.908598 10156 catalog_manager.cc:5495] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 reported cstate change: term changed from 2 to 3, leader changed from df82df23ca924767bfc57e19c09e245a (127.9.84.2) to 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1). New cstate: current_term: 3 leader_uuid: "4932e5163456455ca4b8c5a7736031f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:02.925284 10156 catalog_manager.cc:5495] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 reported cstate change: term changed from 2 to 3, leader changed from df82df23ca924767bfc57e19c09e245a (127.9.84.2) to 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1). New cstate: current_term: 3 leader_uuid: "4932e5163456455ca4b8c5a7736031f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:02.983376 10631 raft_consensus.cc:979] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a: : Instructing follower 4932e5163456455ca4b8c5a7736031f4 to start an election
I20260809 15:51:02.983803 10631 raft_consensus.cc:1067] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 LEADER]: Signalling peer 4932e5163456455ca4b8c5a7736031f4 to start an election
I20260809 15:51:02.985101 10274 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01"
dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
from {username='slave'} at 127.0.0.1:38918
I20260809 15:51:02.985576 10274 raft_consensus.cc:482] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:02.985880 10274 raft_consensus.cc:3037] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:51:02.990455 10274 raft_consensus.cc:504] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:02.991915 10274 leader_election.cc:288] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 4 election: Requested vote from peers df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:51:02.992796 10360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:51:02.993073 10445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
I20260809 15:51:02.993391 10360 raft_consensus.cc:3037] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:51:02.993543 10445 raft_consensus.cc:3032] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:51:02.993840 10445 raft_consensus.cc:729] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Running, Role: LEADER
I20260809 15:51:02.994539 10445 consensus_queue.cc:260] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:02.995746 10445 raft_consensus.cc:3037] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:51:02.999503 10360 raft_consensus.cc:2454] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 4.
I20260809 15:51:03.000590 10238 leader_election.cc:302] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:51:03.001343 10635 raft_consensus.cc:2781] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:51:03.001806 10445 raft_consensus.cc:2454] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 4.
I20260809 15:51:03.001749 10635 raft_consensus.cc:686] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 4 LEADER]: Becoming Leader. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Running, Role: LEADER
I20260809 15:51:03.002929 10635 consensus_queue.cc:237] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:03.009421 10156 catalog_manager.cc:5495] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 reported cstate change: term changed from 3 to 4, leader changed from 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3) to 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1). New cstate: current_term: 4 leader_uuid: "4932e5163456455ca4b8c5a7736031f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:03.138293 10634 raft_consensus.cc:979] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a: : Instructing follower 4932e5163456455ca4b8c5a7736031f4 to start an election
I20260809 15:51:03.138731 10633 raft_consensus.cc:1067] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 2 LEADER]: Signalling peer 4932e5163456455ca4b8c5a7736031f4 to start an election
I20260809 15:51:03.139971 10274 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00c2886e82e647e6986c2ace7f0fb5d2"
dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
from {username='slave'} at 127.0.0.1:38926
I20260809 15:51:03.140422 10274 raft_consensus.cc:482] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:03.140708 10274 raft_consensus.cc:3037] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:51:03.145259 10274 raft_consensus.cc:504] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:03.146667 10274 leader_election.cc:288] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 3 election: Requested vote from peers df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:51:03.147445 10360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00c2886e82e647e6986c2ace7f0fb5d2" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:51:03.147964 10360 raft_consensus.cc:3032] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:51:03.147854 10445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00c2886e82e647e6986c2ace7f0fb5d2" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
I20260809 15:51:03.148278 10360 raft_consensus.cc:729] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 2 LEADER]: Becoming Follower/Learner. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Running, Role: LEADER
I20260809 15:51:03.148464 10445 raft_consensus.cc:3037] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:51:03.149055 10360 consensus_queue.cc:260] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:03.150172 10360 raft_consensus.cc:3037] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:51:03.154304 10445 raft_consensus.cc:2454] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 3.
I20260809 15:51:03.155244 10237 leader_election.cc:302] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, 972dcf60b2944faa9d0ceec67f064e5a; no voters:
I20260809 15:51:03.155670 10360 raft_consensus.cc:2454] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 3.
I20260809 15:51:03.155880 10635 raft_consensus.cc:2781] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:51:03.156324 10635 raft_consensus.cc:686] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 3 LEADER]: Becoming Leader. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Running, Role: LEADER
I20260809 15:51:03.157076 10635 consensus_queue.cc:237] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:03.164047 10156 catalog_manager.cc:5495] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 reported cstate change: term changed from 2 to 3, leader changed from df82df23ca924767bfc57e19c09e245a (127.9.84.2) to 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1). New cstate: current_term: 3 leader_uuid: "4932e5163456455ca4b8c5a7736031f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:03.165053 10646 raft_consensus.cc:979] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a: : Instructing follower 4932e5163456455ca4b8c5a7736031f4 to start an election
I20260809 15:51:03.165485 10631 raft_consensus.cc:1067] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 LEADER]: Signalling peer 4932e5163456455ca4b8c5a7736031f4 to start an election
I20260809 15:51:03.166800 10274 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b5cc2389e9a74343b1e06708819e63c2"
dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
from {username='slave'} at 127.0.0.1:38918
I20260809 15:51:03.167245 10274 raft_consensus.cc:482] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:03.167534 10274 raft_consensus.cc:3037] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:51:03.172240 10274 raft_consensus.cc:504] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:03.173462 10274 leader_election.cc:288] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 4 election: Requested vote from peers df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:51:03.174245 10360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5cc2389e9a74343b1e06708819e63c2" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:51:03.174427 10445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5cc2389e9a74343b1e06708819e63c2" candidate_uuid: "4932e5163456455ca4b8c5a7736031f4" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
I20260809 15:51:03.174696 10360 raft_consensus.cc:3037] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:51:03.174903 10445 raft_consensus.cc:3032] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:51:03.175179 10445 raft_consensus.cc:729] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Running, Role: LEADER
I20260809 15:51:03.175783 10445 consensus_queue.cc:260] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:03.176607 10445 raft_consensus.cc:3037] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:51:03.178792 10360 raft_consensus.cc:2454] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 4.
I20260809 15:51:03.179612 10238 leader_election.cc:302] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4932e5163456455ca4b8c5a7736031f4, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:51:03.180337 10635 raft_consensus.cc:2781] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:51:03.180716 10635 raft_consensus.cc:686] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 4 LEADER]: Becoming Leader. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Running, Role: LEADER
I20260809 15:51:03.180912 10445 raft_consensus.cc:2454] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4932e5163456455ca4b8c5a7736031f4 in term 4.
I20260809 15:51:03.181483 10635 consensus_queue.cc:237] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:03.187670 10156 catalog_manager.cc:5495] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 reported cstate change: term changed from 3 to 4, leader changed from 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3) to 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1). New cstate: current_term: 4 leader_uuid: "4932e5163456455ca4b8c5a7736031f4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:03.346575 10360 raft_consensus.cc:1261] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 3 FOLLOWER]: Refusing update from remote peer 4932e5163456455ca4b8c5a7736031f4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:51:03.348208 10630 consensus_queue.cc:1035] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:03.361311 10445 raft_consensus.cc:1261] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 FOLLOWER]: Refusing update from remote peer 4932e5163456455ca4b8c5a7736031f4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:51:03.362658 10630 consensus_queue.cc:1035] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:03.409541 10445 raft_consensus.cc:1261] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 4 FOLLOWER]: Refusing update from remote peer 4932e5163456455ca4b8c5a7736031f4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:51:03.410696 10630 consensus_queue.cc:1035] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:03.414026 10444 raft_consensus.cc:1261] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 FOLLOWER]: Refusing update from remote peer 4932e5163456455ca4b8c5a7736031f4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:51:03.415268 10630 consensus_queue.cc:1035] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:03.419693 10360 raft_consensus.cc:1261] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 4 FOLLOWER]: Refusing update from remote peer 4932e5163456455ca4b8c5a7736031f4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:51:03.421893 10635 consensus_queue.cc:1035] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:03.431767 10360 raft_consensus.cc:1261] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [term 3 FOLLOWER]: Refusing update from remote peer 4932e5163456455ca4b8c5a7736031f4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:51:03.434888 10639 consensus_queue.cc:1035] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:03.589766 10360 raft_consensus.cc:1261] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 3 FOLLOWER]: Refusing update from remote peer 4932e5163456455ca4b8c5a7736031f4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:51:03.590970 10639 consensus_queue.cc:1035] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:03.599270 10445 raft_consensus.cc:1261] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 FOLLOWER]: Refusing update from remote peer 4932e5163456455ca4b8c5a7736031f4: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:51:03.600719 10639 consensus_queue.cc:1035] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:03.687642 10360 raft_consensus.cc:1261] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [term 4 FOLLOWER]: Refusing update from remote peer 4932e5163456455ca4b8c5a7736031f4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:51:03.688949 10639 consensus_queue.cc:1035] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:03.699330 10445 raft_consensus.cc:1261] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 4 FOLLOWER]: Refusing update from remote peer 4932e5163456455ca4b8c5a7736031f4: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:51:03.700899 10663 consensus_queue.cc:1035] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:05.625645 9552 auto_leader_rebalancer-test.cc:373] The leader distribution is
I20260809 15:51:05.626291 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:05.633162 10274 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "00c2886e82e647e6986c2ace7f0fb5d2"
dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
mode: GRACEFUL
new_leader_uuid: "df82df23ca924767bfc57e19c09e245a"
from {username='slave'} at 127.0.0.1:38952
I20260809 15:51:05.633580 10274 raft_consensus.cc:595] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 3 LEADER]: Received request to transfer leadership to TS df82df23ca924767bfc57e19c09e245a
I20260809 15:51:05.635555 10274 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01"
dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
mode: GRACEFUL
new_leader_uuid: "df82df23ca924767bfc57e19c09e245a"
from {username='slave'} at 127.0.0.1:38952
I20260809 15:51:05.635972 10274 raft_consensus.cc:595] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 4 LEADER]: Received request to transfer leadership to TS df82df23ca924767bfc57e19c09e245a
I20260809 15:51:05.637825 10274 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b118cb45b2544ec68799e11793d4b481"
dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
mode: GRACEFUL
new_leader_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
from {username='slave'} at 127.0.0.1:38952
I20260809 15:51:05.638207 10274 raft_consensus.cc:595] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 3 LEADER]: Received request to transfer leadership to TS 972dcf60b2944faa9d0ceec67f064e5a
I20260809 15:51:05.639932 10274 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b5cc2389e9a74343b1e06708819e63c2"
dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
mode: GRACEFUL
new_leader_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
from {username='slave'} at 127.0.0.1:38952
I20260809 15:51:05.640278 10274 raft_consensus.cc:595] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 4 LEADER]: Received request to transfer leadership to TS 972dcf60b2944faa9d0ceec67f064e5a
I20260809 15:51:05.641052 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260809 15:51:05.641462 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:51:05.647742 10665 raft_consensus.cc:979] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4: : Instructing follower df82df23ca924767bfc57e19c09e245a to start an election
I20260809 15:51:05.648080 10665 raft_consensus.cc:1067] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 4 LEADER]: Signalling peer df82df23ca924767bfc57e19c09e245a to start an election
I20260809 15:51:05.649226 10360 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01"
dest_uuid: "df82df23ca924767bfc57e19c09e245a"
from {username='slave'} at 127.0.0.1:41376
I20260809 15:51:05.649642 10360 raft_consensus.cc:482] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:05.649950 10360 raft_consensus.cc:3037] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:51:05.653983 10360 raft_consensus.cc:504] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:05.655685 10360 leader_election.cc:288] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 5 election: Requested vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:51:05.656632 10445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
I20260809 15:51:05.656880 10274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c7503a4f3ba42d7b51a0fc1f1521c01" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
I20260809 15:51:05.657182 10445 raft_consensus.cc:3037] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:51:05.657331 10274 raft_consensus.cc:3032] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:51:05.657639 10274 raft_consensus.cc:729] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Running, Role: LEADER
I20260809 15:51:05.658298 10274 consensus_queue.cc:260] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:05.659451 10274 raft_consensus.cc:3037] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:51:05.661063 10666 raft_consensus.cc:979] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4: : Instructing follower df82df23ca924767bfc57e19c09e245a to start an election
I20260809 15:51:05.661533 10665 raft_consensus.cc:1067] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 3 LEADER]: Signalling peer df82df23ca924767bfc57e19c09e245a to start an election
I20260809 15:51:05.662757 10360 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00c2886e82e647e6986c2ace7f0fb5d2"
dest_uuid: "df82df23ca924767bfc57e19c09e245a"
from {username='slave'} at 127.0.0.1:41376
I20260809 15:51:05.663167 10360 raft_consensus.cc:482] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:05.663050 10445 raft_consensus.cc:2454] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df82df23ca924767bfc57e19c09e245a in term 5.
I20260809 15:51:05.663584 10360 raft_consensus.cc:3037] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:51:05.664454 10321 leader_election.cc:302] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 972dcf60b2944faa9d0ceec67f064e5a, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:51:05.665247 10633 raft_consensus.cc:2781] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:51:05.665422 10274 raft_consensus.cc:2454] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df82df23ca924767bfc57e19c09e245a in term 5.
I20260809 15:51:05.666368 10633 raft_consensus.cc:686] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 5 LEADER]: Becoming Leader. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Running, Role: LEADER
I20260809 15:51:05.667169 10633 consensus_queue.cc:237] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:05.669710 10360 raft_consensus.cc:504] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:05.671754 10360 leader_election.cc:288] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 4 election: Requested vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3:44211)
I20260809 15:51:05.672062 10274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00c2886e82e647e6986c2ace7f0fb5d2" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
I20260809 15:51:05.672333 10445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00c2886e82e647e6986c2ace7f0fb5d2" candidate_uuid: "df82df23ca924767bfc57e19c09e245a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
I20260809 15:51:05.672642 10274 raft_consensus.cc:3032] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:51:05.672876 10445 raft_consensus.cc:3037] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:51:05.672941 10274 raft_consensus.cc:729] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Running, Role: LEADER
I20260809 15:51:05.673688 10274 consensus_queue.cc:260] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:05.674789 10274 raft_consensus.cc:3037] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:51:05.675012 10156 catalog_manager.cc:5495] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a reported cstate change: term changed from 4 to 5, leader changed from 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1) to df82df23ca924767bfc57e19c09e245a (127.9.84.2). New cstate: current_term: 5 leader_uuid: "df82df23ca924767bfc57e19c09e245a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:05.676923 10445 raft_consensus.cc:2454] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df82df23ca924767bfc57e19c09e245a in term 4.
I20260809 15:51:05.677692 10321 leader_election.cc:302] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 972dcf60b2944faa9d0ceec67f064e5a, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:51:05.678521 10633 raft_consensus.cc:2781] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:51:05.679026 10633 raft_consensus.cc:686] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 4 LEADER]: Becoming Leader. State: Replica: df82df23ca924767bfc57e19c09e245a, State: Running, Role: LEADER
I20260809 15:51:05.679777 10633 consensus_queue.cc:237] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:05.680538 10274 raft_consensus.cc:2454] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df82df23ca924767bfc57e19c09e245a in term 4.
I20260809 15:51:05.690946 10156 catalog_manager.cc:5495] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a reported cstate change: term changed from 3 to 4, leader changed from 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1) to df82df23ca924767bfc57e19c09e245a (127.9.84.2). New cstate: current_term: 4 leader_uuid: "df82df23ca924767bfc57e19c09e245a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:05.725235 10666 raft_consensus.cc:979] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4: : Instructing follower 972dcf60b2944faa9d0ceec67f064e5a to start an election
I20260809 15:51:05.725760 10665 raft_consensus.cc:1067] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 3 LEADER]: Signalling peer 972dcf60b2944faa9d0ceec67f064e5a to start an election
I20260809 15:51:05.727018 10445 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b118cb45b2544ec68799e11793d4b481"
dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
from {username='slave'} at 127.0.0.1:39714
I20260809 15:51:05.727444 10445 raft_consensus.cc:482] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:05.727666 10445 raft_consensus.cc:3037] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:51:05.731940 10445 raft_consensus.cc:504] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:05.733587 10445 leader_election.cc:288] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 4 election: Requested vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197)
I20260809 15:51:05.734334 10274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b118cb45b2544ec68799e11793d4b481" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
I20260809 15:51:05.734565 10360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b118cb45b2544ec68799e11793d4b481" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:51:05.734884 10274 raft_consensus.cc:3032] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:51:05.735097 10360 raft_consensus.cc:3037] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:51:05.735333 10274 raft_consensus.cc:729] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Running, Role: LEADER
I20260809 15:51:05.736011 10274 consensus_queue.cc:260] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:05.736799 10274 raft_consensus.cc:3037] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:51:05.739590 10360 raft_consensus.cc:2454] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 4.
I20260809 15:51:05.740571 10406 leader_election.cc:302] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 972dcf60b2944faa9d0ceec67f064e5a, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:51:05.741201 10631 raft_consensus.cc:2781] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:51:05.741163 10274 raft_consensus.cc:2454] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 4.
I20260809 15:51:05.742714 10631 raft_consensus.cc:686] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [term 4 LEADER]: Becoming Leader. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Running, Role: LEADER
I20260809 15:51:05.743495 10631 consensus_queue.cc:237] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:05.749454 10156 catalog_manager.cc:5495] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a reported cstate change: term changed from 3 to 4, leader changed from 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1) to 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3). New cstate: current_term: 4 leader_uuid: "972dcf60b2944faa9d0ceec67f064e5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:06.087671 10274 raft_consensus.cc:1261] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 5 FOLLOWER]: Refusing update from remote peer df82df23ca924767bfc57e19c09e245a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260809 15:51:06.088716 10445 raft_consensus.cc:1261] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 5 FOLLOWER]: Refusing update from remote peer df82df23ca924767bfc57e19c09e245a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260809 15:51:06.088858 10633 consensus_queue.cc:1035] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:06.089978 10668 consensus_queue.cc:1035] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [LEADER]: Connected to new peer: Peer: permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:06.109457 10666 raft_consensus.cc:979] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4: : Instructing follower 972dcf60b2944faa9d0ceec67f064e5a to start an election
I20260809 15:51:06.110018 10665 raft_consensus.cc:1067] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 4 LEADER]: Signalling peer 972dcf60b2944faa9d0ceec67f064e5a to start an election
I20260809 15:51:06.111459 10445 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b5cc2389e9a74343b1e06708819e63c2"
dest_uuid: "972dcf60b2944faa9d0ceec67f064e5a"
from {username='slave'} at 127.0.0.1:39714
I20260809 15:51:06.111889 10445 raft_consensus.cc:482] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:06.112159 10445 raft_consensus.cc:3037] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:51:06.116650 10445 raft_consensus.cc:504] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:06.117942 10445 leader_election.cc:288] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 5 election: Requested vote from peers 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561), df82df23ca924767bfc57e19c09e245a (127.9.84.2:35197)
I20260809 15:51:06.118713 10274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5cc2389e9a74343b1e06708819e63c2" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "4932e5163456455ca4b8c5a7736031f4"
I20260809 15:51:06.118901 10360 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5cc2389e9a74343b1e06708819e63c2" candidate_uuid: "972dcf60b2944faa9d0ceec67f064e5a" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "df82df23ca924767bfc57e19c09e245a"
I20260809 15:51:06.119194 10274 raft_consensus.cc:3032] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:51:06.119482 10360 raft_consensus.cc:3037] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:51:06.119514 10274 raft_consensus.cc:729] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 4932e5163456455ca4b8c5a7736031f4, State: Running, Role: LEADER
I20260809 15:51:06.120314 10274 consensus_queue.cc:260] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:06.121274 10274 raft_consensus.cc:3037] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:51:06.123651 10360 raft_consensus.cc:2454] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 5.
I20260809 15:51:06.124434 10406 leader_election.cc:302] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 972dcf60b2944faa9d0ceec67f064e5a, df82df23ca924767bfc57e19c09e245a; no voters:
I20260809 15:51:06.125039 10631 raft_consensus.cc:2781] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:51:06.125540 10274 raft_consensus.cc:2454] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 972dcf60b2944faa9d0ceec67f064e5a in term 5.
I20260809 15:51:06.125639 10631 raft_consensus.cc:686] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 5 LEADER]: Becoming Leader. State: Replica: 972dcf60b2944faa9d0ceec67f064e5a, State: Running, Role: LEADER
I20260809 15:51:06.126425 10631 consensus_queue.cc:237] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } }
I20260809 15:51:06.132869 10156 catalog_manager.cc:5495] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a reported cstate change: term changed from 4 to 5, leader changed from 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1) to 972dcf60b2944faa9d0ceec67f064e5a (127.9.84.3). New cstate: current_term: 5 leader_uuid: "972dcf60b2944faa9d0ceec67f064e5a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:06.205842 10274 raft_consensus.cc:1261] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 4 FOLLOWER]: Refusing update from remote peer df82df23ca924767bfc57e19c09e245a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:51:06.206941 10680 consensus_queue.cc:1035] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:06.214766 10445 raft_consensus.cc:1261] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 4 FOLLOWER]: Refusing update from remote peer df82df23ca924767bfc57e19c09e245a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:51:06.216092 10633 consensus_queue.cc:1035] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [LEADER]: Connected to new peer: Peer: permanent_uuid: "972dcf60b2944faa9d0ceec67f064e5a" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 44211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:06.330477 10360 raft_consensus.cc:1261] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 4 FOLLOWER]: Refusing update from remote peer 972dcf60b2944faa9d0ceec67f064e5a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:51:06.331549 10631 consensus_queue.cc:1035] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:06.339758 10273 raft_consensus.cc:1261] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 4 FOLLOWER]: Refusing update from remote peer 972dcf60b2944faa9d0ceec67f064e5a: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:51:06.341598 10676 consensus_queue.cc:1035] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:06.642313 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:06.647444 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:51:06.647812 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:51:06.666397 10273 raft_consensus.cc:1261] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 5 FOLLOWER]: Refusing update from remote peer 972dcf60b2944faa9d0ceec67f064e5a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260809 15:51:06.667513 10631 consensus_queue.cc:1035] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "4932e5163456455ca4b8c5a7736031f4" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 40561 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:06.675290 10360 raft_consensus.cc:1261] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [term 5 FOLLOWER]: Refusing update from remote peer 972dcf60b2944faa9d0ceec67f064e5a: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260809 15:51:06.676712 10676 consensus_queue.cc:1035] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [LEADER]: Connected to new peer: Peer: permanent_uuid: "df82df23ca924767bfc57e19c09e245a" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 35197 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:07.649031 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:07.655793 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:51:07.656265 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:51:08.657034 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:08.661725 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:51:08.662194 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:51:09.662963 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:09.668488 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:51:09.668882 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:51:10.669798 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:10.674701 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:51:10.675082 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:51:11.675983 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:11.680912 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:51:11.681289 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:51:12.682029 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:12.686980 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:51:12.687373 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:51:13.688299 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:13.693259 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:51:13.693645 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:51:14.694572 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:14.700058 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 0
I20260809 15:51:14.700433 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:51:15.733243 9552 auto_leader_rebalancer-test.cc:384] The leader distribution is
I20260809 15:51:15.733893 9552 tablet_server.cc:178] TabletServer@127.9.84.1:0 shutting down...
W20260809 15:51:15.760372 10316 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer [suppressed 5 similar messages]
W20260809 15:51:15.764328 10316 consensus_peers.cc:487] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a -> Peer 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561): Couldn't send request to peer 4932e5163456455ca4b8c5a7736031f4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on TabletServer. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:15.766641 9552 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260809 15:51:15.768074 9552 tablet_replica.cc:330] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4: stopping tablet replica
I20260809 15:51:15.769120 9552 raft_consensus.cc:2229] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:15.769999 9552 raft_consensus.cc:2258] T b31c1906f58b40d48decf694093d62a3 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.772562 9552 tablet_replica.cc:330] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4: stopping tablet replica
I20260809 15:51:15.773150 9552 raft_consensus.cc:2229] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:51:15.774012 9552 raft_consensus.cc:2258] T 5ec90e88181c4e5085d12f183609cbae P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.776048 9552 tablet_replica.cc:330] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4: stopping tablet replica
I20260809 15:51:15.776582 9552 raft_consensus.cc:2229] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:15.776975 9552 raft_consensus.cc:2258] T b5cc2389e9a74343b1e06708819e63c2 P 4932e5163456455ca4b8c5a7736031f4 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.778697 9552 tablet_replica.cc:330] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4: stopping tablet replica
I20260809 15:51:15.779168 9552 raft_consensus.cc:2229] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [term 3 LEADER]: Raft consensus shutting down.
I20260809 15:51:15.779933 9552 raft_consensus.cc:2258] T a4ff800c08cd44158fc25d6fd4735dc2 P 4932e5163456455ca4b8c5a7736031f4 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.781490 9552 tablet_replica.cc:330] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4: stopping tablet replica
I20260809 15:51:15.781973 9552 raft_consensus.cc:2229] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [term 2 LEADER]: Raft consensus shutting down.
I20260809 15:51:15.782694 9552 raft_consensus.cc:2258] T eb07891a9e804c51a61318118a0e8886 P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.784245 9552 tablet_replica.cc:330] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4: stopping tablet replica
I20260809 15:51:15.784714 9552 raft_consensus.cc:2229] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:15.785116 9552 raft_consensus.cc:2258] T b118cb45b2544ec68799e11793d4b481 P 4932e5163456455ca4b8c5a7736031f4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.787086 9552 tablet_replica.cc:330] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4: stopping tablet replica
I20260809 15:51:15.787717 9552 raft_consensus.cc:2229] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:15.788092 9552 raft_consensus.cc:2258] T 00c2886e82e647e6986c2ace7f0fb5d2 P 4932e5163456455ca4b8c5a7736031f4 [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.789633 9552 tablet_replica.cc:330] T cc8769d170ef4bd089200728cd5f52cf P 4932e5163456455ca4b8c5a7736031f4: stopping tablet replica
I20260809 15:51:15.790028 9552 raft_consensus.cc:2229] T cc8769d170ef4bd089200728cd5f52cf P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:15.790465 9552 raft_consensus.cc:2258] T cc8769d170ef4bd089200728cd5f52cf P 4932e5163456455ca4b8c5a7736031f4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.791953 9552 tablet_replica.cc:330] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4: stopping tablet replica
I20260809 15:51:15.792336 9552 raft_consensus.cc:2229] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [term 2 LEADER]: Raft consensus shutting down.
I20260809 15:51:15.792985 9552 raft_consensus.cc:2258] T b0af688bc961452abb333bd49c4c595d P 4932e5163456455ca4b8c5a7736031f4 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.794461 9552 tablet_replica.cc:330] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4: stopping tablet replica
I20260809 15:51:15.794850 9552 raft_consensus.cc:2229] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 5 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:15.795238 9552 raft_consensus.cc:2258] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 4932e5163456455ca4b8c5a7736031f4 [term 5 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.823827 9552 tablet_server.cc:194] TabletServer@127.9.84.1:0 shutdown complete.
I20260809 15:51:15.843108 9552 tablet_server.cc:178] TabletServer@127.9.84.2:0 shutting down...
I20260809 15:51:15.869184 9552 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260809 15:51:15.869781 9552 tablet_replica.cc:330] T eb07891a9e804c51a61318118a0e8886 P df82df23ca924767bfc57e19c09e245a: stopping tablet replica
I20260809 15:51:15.870553 9552 raft_consensus.cc:2229] T eb07891a9e804c51a61318118a0e8886 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:15.871045 9552 raft_consensus.cc:2258] T eb07891a9e804c51a61318118a0e8886 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.873271 9552 tablet_replica.cc:330] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a: stopping tablet replica
I20260809 15:51:15.874126 9552 raft_consensus.cc:2229] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:15.874707 9552 raft_consensus.cc:2258] T a4ff800c08cd44158fc25d6fd4735dc2 P df82df23ca924767bfc57e19c09e245a [term 3 FOLLOWER]: Raft consensus is shut down!
W20260809 15:51:15.876672 10316 consensus_peers.cc:487] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a -> Peer 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561): Couldn't send request to peer 4932e5163456455ca4b8c5a7736031f4. Status: Network error: Client connection negotiation failed: client connection to 127.9.84.1:40561: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:15.876746 9552 tablet_replica.cc:330] T b31c1906f58b40d48decf694093d62a3 P df82df23ca924767bfc57e19c09e245a: stopping tablet replica
W20260809 15:51:15.877084 10316 consensus_peers.cc:487] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a -> Peer 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561): Couldn't send request to peer 4932e5163456455ca4b8c5a7736031f4. Status: Network error: Client connection negotiation failed: client connection to 127.9.84.1:40561: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:15.877400 9552 raft_consensus.cc:2229] T b31c1906f58b40d48decf694093d62a3 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:15.877835 9552 raft_consensus.cc:2258] T b31c1906f58b40d48decf694093d62a3 P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.879580 9552 tablet_replica.cc:330] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a: stopping tablet replica
I20260809 15:51:15.880100 9552 raft_consensus.cc:2229] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 4 LEADER]: Raft consensus shutting down.
I20260809 15:51:15.880854 9552 raft_consensus.cc:2258] T 00c2886e82e647e6986c2ace7f0fb5d2 P df82df23ca924767bfc57e19c09e245a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.882536 9552 tablet_replica.cc:330] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a: stopping tablet replica
I20260809 15:51:15.883010 9552 raft_consensus.cc:2229] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:15.883416 9552 raft_consensus.cc:2258] T b118cb45b2544ec68799e11793d4b481 P df82df23ca924767bfc57e19c09e245a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.884981 9552 tablet_replica.cc:330] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a: stopping tablet replica
I20260809 15:51:15.885433 9552 raft_consensus.cc:2229] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:15.885809 9552 raft_consensus.cc:2258] T b0af688bc961452abb333bd49c4c595d P df82df23ca924767bfc57e19c09e245a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.887367 9552 tablet_replica.cc:330] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a: stopping tablet replica
I20260809 15:51:15.887827 9552 raft_consensus.cc:2229] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:51:15.888563 9552 raft_consensus.cc:2258] T cc8769d170ef4bd089200728cd5f52cf P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.890136 9552 tablet_replica.cc:330] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a: stopping tablet replica
I20260809 15:51:15.890604 9552 raft_consensus.cc:2229] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 5 LEADER]: Raft consensus shutting down.
I20260809 15:51:15.891299 9552 raft_consensus.cc:2258] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P df82df23ca924767bfc57e19c09e245a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.892895 9552 tablet_replica.cc:330] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a: stopping tablet replica
I20260809 15:51:15.893348 9552 raft_consensus.cc:2229] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:15.893702 9552 raft_consensus.cc:2258] T 5ec90e88181c4e5085d12f183609cbae P df82df23ca924767bfc57e19c09e245a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.895306 9552 tablet_replica.cc:330] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a: stopping tablet replica
I20260809 15:51:15.895728 9552 raft_consensus.cc:2229] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:15.896200 9552 raft_consensus.cc:2258] T b5cc2389e9a74343b1e06708819e63c2 P df82df23ca924767bfc57e19c09e245a [term 5 FOLLOWER]: Raft consensus is shut down!
W20260809 15:51:15.909235 10401 consensus_peers.cc:487] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a -> Peer 4932e5163456455ca4b8c5a7736031f4 (127.9.84.1:40561): Couldn't send request to peer 4932e5163456455ca4b8c5a7736031f4. Status: Network error: Client connection negotiation failed: client connection to 127.9.84.1:40561: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:15.924360 9552 tablet_server.cc:194] TabletServer@127.9.84.2:0 shutdown complete.
I20260809 15:51:15.942512 9552 tablet_server.cc:178] TabletServer@127.9.84.3:0 shutting down...
I20260809 15:51:15.967597 9552 ts_tablet_manager.cc:1500] Shutting down tablet manager...
I20260809 15:51:15.968124 9552 tablet_replica.cc:330] T 5ec90e88181c4e5085d12f183609cbae P 972dcf60b2944faa9d0ceec67f064e5a: stopping tablet replica
I20260809 15:51:15.968667 9552 raft_consensus.cc:2229] T 5ec90e88181c4e5085d12f183609cbae P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:15.969043 9552 raft_consensus.cc:2258] T 5ec90e88181c4e5085d12f183609cbae P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.970768 9552 tablet_replica.cc:330] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a: stopping tablet replica
I20260809 15:51:15.971195 9552 raft_consensus.cc:2229] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 5 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:15.971551 9552 raft_consensus.cc:2258] T 4c7503a4f3ba42d7b51a0fc1f1521c01 P 972dcf60b2944faa9d0ceec67f064e5a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.972982 9552 tablet_replica.cc:330] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a: stopping tablet replica
I20260809 15:51:15.973446 9552 raft_consensus.cc:2229] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:15.973809 9552 raft_consensus.cc:2258] T eb07891a9e804c51a61318118a0e8886 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.975268 9552 tablet_replica.cc:330] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a: stopping tablet replica
I20260809 15:51:15.975652 9552 raft_consensus.cc:2229] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:15.975979 9552 raft_consensus.cc:2258] T a4ff800c08cd44158fc25d6fd4735dc2 P 972dcf60b2944faa9d0ceec67f064e5a [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.977345 9552 tablet_replica.cc:330] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a: stopping tablet replica
I20260809 15:51:15.977744 9552 raft_consensus.cc:2229] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 4 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:15.978192 9552 raft_consensus.cc:2258] T 00c2886e82e647e6986c2ace7f0fb5d2 P 972dcf60b2944faa9d0ceec67f064e5a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.979667 9552 tablet_replica.cc:330] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a: stopping tablet replica
I20260809 15:51:15.980093 9552 raft_consensus.cc:2229] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [term 4 LEADER]: Raft consensus shutting down.
I20260809 15:51:15.980751 9552 raft_consensus.cc:2258] T b118cb45b2544ec68799e11793d4b481 P 972dcf60b2944faa9d0ceec67f064e5a [term 4 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.982254 9552 tablet_replica.cc:330] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a: stopping tablet replica
I20260809 15:51:15.982681 9552 raft_consensus.cc:2229] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 5 LEADER]: Raft consensus shutting down.
I20260809 15:51:15.983323 9552 raft_consensus.cc:2258] T b5cc2389e9a74343b1e06708819e63c2 P 972dcf60b2944faa9d0ceec67f064e5a [term 5 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.984854 9552 tablet_replica.cc:330] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a: stopping tablet replica
I20260809 15:51:15.985378 9552 raft_consensus.cc:2229] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 LEADER]: Raft consensus shutting down.
I20260809 15:51:15.986168 9552 raft_consensus.cc:2258] T b31c1906f58b40d48decf694093d62a3 P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.987741 9552 tablet_replica.cc:330] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a: stopping tablet replica
I20260809 15:51:15.988193 9552 raft_consensus.cc:2229] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:15.988611 9552 raft_consensus.cc:2258] T b0af688bc961452abb333bd49c4c595d P 972dcf60b2944faa9d0ceec67f064e5a [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:15.990214 9552 tablet_replica.cc:330] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a: stopping tablet replica
I20260809 15:51:15.990648 9552 raft_consensus.cc:2229] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:15.991007 9552 raft_consensus.cc:2258] T cc8769d170ef4bd089200728cd5f52cf P 972dcf60b2944faa9d0ceec67f064e5a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:16.019641 9552 tablet_server.cc:194] TabletServer@127.9.84.3:0 shutdown complete.
I20260809 15:51:16.038686 9552 master.cc:537] Master@127.9.84.62:37411 shutting down...
I20260809 15:51:16.313300 9552 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14 [term 1 LEADER]: Raft consensus shutting down.
I20260809 15:51:16.313848 9552 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:16.314236 9552 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9be32ac118be402fa78d175a5e66ff14: stopping tablet replica
I20260809 15:51:16.333278 9552 master.cc:552] Master@127.9.84.62:37411 shutdown complete.
4932e5163456455ca4b8c5a7736031f4 5
972dcf60b2944faa9d0ceec67f064e5a 1
df82df23ca924767bfc57e19c09e245a 4
4932e5163456455ca4b8c5a7736031f4 3
972dcf60b2944faa9d0ceec67f064e5a 3
df82df23ca924767bfc57e19c09e245a 4
4932e5163456455ca4b8c5a7736031f4 8
972dcf60b2944faa9d0ceec67f064e5a 1
df82df23ca924767bfc57e19c09e245a 1
4932e5163456455ca4b8c5a7736031f4 4
972dcf60b2944faa9d0ceec67f064e5a 3
df82df23ca924767bfc57e19c09e245a 3
[ OK ] LeaderRebalancerTest.FunctionalTestForNotDivided (28766 ms)
[ RUN ] LeaderRebalancerTest.AddTserver
I20260809 15:51:16.373986 9552 internal_mini_cluster.cc:156] Creating distributed mini masters. Addrs: 127.9.84.62:42773
I20260809 15:51:16.375036 9552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:51:16.380519 10709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:51:16.380520 10710 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:51:16.382201 10712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:51:16.382936 9552 server_base.cc:1034] running on GCE node
I20260809 15:51:16.383893 9552 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 15:51:16.384109 9552 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 15:51:16.384261 9552 hybrid_clock.cc:648] HybridClock initialized: now 1786290676384246 us; error 0 us; skew 500 ppm
I20260809 15:51:16.384748 9552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:51:16.387403 9552 webserver.cc:460] Webserver started at http://127.9.84.62:44647/ using document root <none> and password file <none>
I20260809 15:51:16.387874 9552 fs_manager.cc:359] Metadata directory not provided
I20260809 15:51:16.388074 9552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:51:16.388351 9552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:51:16.389560 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/master-0-root/instance:
uuid: "9adabcb74b524f959359b60ec0f8486f"
format_stamp: "Formatted at 2026-08-09 15:51:16 on dist-test-slave-mn4r"
I20260809 15:51:16.394009 9552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 15:51:16.397161 10717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:51:16.397888 9552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260809 15:51:16.398188 9552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/master-0-root
uuid: "9adabcb74b524f959359b60ec0f8486f"
format_stamp: "Formatted at 2026-08-09 15:51:16 on dist-test-slave-mn4r"
I20260809 15:51:16.398438 9552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/master-0-root
metadata directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/master-0-root
1 data directories: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/master-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:51:16.425046 9552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:51:16.426287 9552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:51:16.464097 9552 rpc_server.cc:307] RPC server started. Bound to: 127.9.84.62:42773
I20260809 15:51:16.464165 10773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.84.62:42773 every 8 connection(s)
I20260809 15:51:16.467856 10774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:16.477248 10774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f: Bootstrap starting.
I20260809 15:51:16.481451 10774 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:16.485396 10774 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f: No bootstrap required, opened a new log
I20260809 15:51:16.487591 10774 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9adabcb74b524f959359b60ec0f8486f" member_type: VOTER }
I20260809 15:51:16.488191 10774 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:16.488430 10774 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9adabcb74b524f959359b60ec0f8486f, State: Initialized, Role: FOLLOWER
I20260809 15:51:16.489004 10774 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9adabcb74b524f959359b60ec0f8486f" member_type: VOTER }
I20260809 15:51:16.489444 10774 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 15:51:16.489694 10774 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 15:51:16.489984 10774 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:16.493503 10774 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9adabcb74b524f959359b60ec0f8486f" member_type: VOTER }
I20260809 15:51:16.494002 10774 leader_election.cc:302] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9adabcb74b524f959359b60ec0f8486f; no voters:
I20260809 15:51:16.495153 10774 leader_election.cc:288] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 15:51:16.495558 10779 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:16.496840 10779 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f [term 1 LEADER]: Becoming Leader. State: Replica: 9adabcb74b524f959359b60ec0f8486f, State: Running, Role: LEADER
I20260809 15:51:16.497498 10779 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9adabcb74b524f959359b60ec0f8486f" member_type: VOTER }
I20260809 15:51:16.498016 10774 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 15:51:16.502341 10780 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9adabcb74b524f959359b60ec0f8486f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9adabcb74b524f959359b60ec0f8486f" member_type: VOTER } }
I20260809 15:51:16.503314 10781 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9adabcb74b524f959359b60ec0f8486f. Latest consensus state: current_term: 1 leader_uuid: "9adabcb74b524f959359b60ec0f8486f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9adabcb74b524f959359b60ec0f8486f" member_type: VOTER } }
I20260809 15:51:16.503800 10780 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f [sys.catalog]: This master's current role is: LEADER
I20260809 15:51:16.503935 10781 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f [sys.catalog]: This master's current role is: LEADER
I20260809 15:51:16.505797 10787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 15:51:16.513669 10787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 15:51:16.517347 9552 internal_mini_cluster.cc:184] Waiting to initialize catalog manager on master 0
I20260809 15:51:16.523444 10787 catalog_manager.cc:1347] Generated new cluster ID: 29f34f6e399f4516a5a253a4cacce68f
I20260809 15:51:16.523689 10787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 15:51:16.588665 10787 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 15:51:16.590255 10787 catalog_manager.cc:1504] Loading token signing keys...
I20260809 15:51:16.602283 10787 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f: Generated new TSK 0
I20260809 15:51:16.602864 10787 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 15:51:16.648592 9552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:51:16.654295 10798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:51:16.655267 10800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:51:16.656733 10802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:51:16.657747 9552 server_base.cc:1034] running on GCE node
I20260809 15:51:16.658727 9552 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 15:51:16.658936 9552 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 15:51:16.659111 9552 hybrid_clock.cc:648] HybridClock initialized: now 1786290676659077 us; error 0 us; skew 500 ppm
I20260809 15:51:16.659626 9552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:51:16.661764 9552 webserver.cc:460] Webserver started at http://127.9.84.1:45415/ using document root <none> and password file <none>
I20260809 15:51:16.662341 9552 fs_manager.cc:359] Metadata directory not provided
I20260809 15:51:16.662611 9552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:51:16.662881 9552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:51:16.663949 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-0-root/instance:
uuid: "a53441872c0c416b8e2f26f52399f536"
format_stamp: "Formatted at 2026-08-09 15:51:16 on dist-test-slave-mn4r"
I20260809 15:51:16.668416 9552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 15:51:16.671497 10807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:51:16.672237 9552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 15:51:16.672528 9552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-0-root
uuid: "a53441872c0c416b8e2f26f52399f536"
format_stamp: "Formatted at 2026-08-09 15:51:16 on dist-test-slave-mn4r"
I20260809 15:51:16.672801 9552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-0-root
metadata directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-0-root
1 data directories: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-0-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:51:16.693526 9552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:51:16.694784 9552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:51:16.696271 9552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:51:16.698688 9552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:51:16.698855 9552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:51:16.699133 9552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:51:16.699370 9552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:51:16.738354 9552 rpc_server.cc:307] RPC server started. Bound to: 127.9.84.1:46615
I20260809 15:51:16.738456 10875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.84.1:46615 every 8 connection(s)
I20260809 15:51:16.742918 9552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:51:16.750401 10881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:51:16.752396 10882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:51:16.753261 10884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:51:16.754573 9552 server_base.cc:1034] running on GCE node
I20260809 15:51:16.756506 9552 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 15:51:16.756783 9552 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 15:51:16.756969 9552 hybrid_clock.cc:648] HybridClock initialized: now 1786290676756953 us; error 0 us; skew 500 ppm
I20260809 15:51:16.757637 9552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:51:16.757855 10877 heartbeater.cc:347] Connected to a master server at 127.9.84.62:42773
I20260809 15:51:16.758265 10877 heartbeater.cc:464] Registering TS with master...
I20260809 15:51:16.758924 10877 heartbeater.cc:511] Master 127.9.84.62:42773 requested a full tablet report, sending...
I20260809 15:51:16.760557 9552 webserver.cc:460] Webserver started at http://127.9.84.2:44931/ using document root <none> and password file <none>
I20260809 15:51:16.760850 10737 ts_manager.cc:194] Registered new tserver with Master: a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615)
I20260809 15:51:16.761099 9552 fs_manager.cc:359] Metadata directory not provided
I20260809 15:51:16.761401 9552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:51:16.761683 9552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:51:16.762960 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-1-root/instance:
uuid: "20730d75ad954a82b126148f8152a9f3"
format_stamp: "Formatted at 2026-08-09 15:51:16 on dist-test-slave-mn4r"
I20260809 15:51:16.763052 10737 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33038
I20260809 15:51:16.767361 9552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260809 15:51:16.770586 10889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:51:16.771265 9552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260809 15:51:16.771559 9552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-1-root
uuid: "20730d75ad954a82b126148f8152a9f3"
format_stamp: "Formatted at 2026-08-09 15:51:16 on dist-test-slave-mn4r"
I20260809 15:51:16.771828 9552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-1-root
metadata directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-1-root
1 data directories: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-1-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:51:16.789840 9552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:51:16.790982 9552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:51:16.792604 9552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:51:16.794704 9552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:51:16.794872 9552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:51:16.795135 9552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:51:16.795280 9552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:51:16.834380 9552 rpc_server.cc:307] RPC server started. Bound to: 127.9.84.2:32783
I20260809 15:51:16.834481 10959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.84.2:32783 every 8 connection(s)
I20260809 15:51:16.838920 9552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:51:16.845983 10963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:51:16.849000 10964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:51:16.852180 10966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:51:16.852337 9552 server_base.cc:1034] running on GCE node
I20260809 15:51:16.853394 9552 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 15:51:16.853669 9552 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 15:51:16.853886 9552 hybrid_clock.cc:648] HybridClock initialized: now 1786290676853853 us; error 0 us; skew 500 ppm
I20260809 15:51:16.854454 9552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:51:16.854796 10960 heartbeater.cc:347] Connected to a master server at 127.9.84.62:42773
I20260809 15:51:16.855228 10960 heartbeater.cc:464] Registering TS with master...
I20260809 15:51:16.856043 10960 heartbeater.cc:511] Master 127.9.84.62:42773 requested a full tablet report, sending...
I20260809 15:51:16.857061 9552 webserver.cc:460] Webserver started at http://127.9.84.3:37575/ using document root <none> and password file <none>
I20260809 15:51:16.857609 9552 fs_manager.cc:359] Metadata directory not provided
I20260809 15:51:16.857822 9552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:51:16.858152 9552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:51:16.858125 10737 ts_manager.cc:194] Registered new tserver with Master: 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:16.859439 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-2-root/instance:
uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
format_stamp: "Formatted at 2026-08-09 15:51:16 on dist-test-slave-mn4r"
I20260809 15:51:16.859797 10737 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33054
I20260809 15:51:16.864049 9552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 15:51:16.867286 10973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:51:16.867998 9552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260809 15:51:16.868276 9552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-2-root
uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
format_stamp: "Formatted at 2026-08-09 15:51:16 on dist-test-slave-mn4r"
I20260809 15:51:16.868553 9552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-2-root
metadata directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-2-root
1 data directories: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-2-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:51:16.892251 9552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:51:16.893349 9552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:51:16.894753 9552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:51:16.896978 9552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:51:16.897197 9552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:51:16.897420 9552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:51:16.897593 9552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:51:16.936739 9552 rpc_server.cc:307] RPC server started. Bound to: 127.9.84.3:39223
I20260809 15:51:16.936839 11047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.84.3:39223 every 8 connection(s)
I20260809 15:51:16.951185 11048 heartbeater.cc:347] Connected to a master server at 127.9.84.62:42773
I20260809 15:51:16.951531 11048 heartbeater.cc:464] Registering TS with master...
I20260809 15:51:16.952211 11048 heartbeater.cc:511] Master 127.9.84.62:42773 requested a full tablet report, sending...
I20260809 15:51:16.953918 10737 ts_manager.cc:194] Registered new tserver with Master: d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:16.954227 9552 internal_mini_cluster.cc:371] 3 TS(s) registered with all masters after 0.014571828s
I20260809 15:51:16.956066 10737 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:33062
I20260809 15:51:16.956861 9552 test_util.cc:268] Using random seed: 893648553
I20260809 15:51:16.980839 10737 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33066:
name: "test-workload"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "int_val"
type: INT32
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "string_val"
type: STRING
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}""\004\001\000\313c+\002\004\001\000\226\307V\004\004\001\000a+\202\006\004\001\000,\217\255\010\004\001\000\367\362\330\n\004\001\000\302V\004\r\004\001\000\215\272/\017\004\001\000X\036[\021\004\001\000#\202\206\023\004\001\000\356\345\261\025\004\001\000\271I\335\027\004\001\000\204\255\010\032\004\001\000O\0214\034\004\001\000\032u_\036\004\001\000\345\330\212 \004\001\000\260<\266\"\004\001\000{\240\341$\004\001\000F\004\r\'\004\001\000\021h8)\004\001\000\334\313c+\004\001\000\247/\217-\004\001\000r\223\272/\004\001\000=\367\3451\004\001\000\010[\0214\004\001\000\323\276<6\004\001\000\236\"h8\004\001\000i\206\223:\004\001\0004\352\276<\004\001\000\377M\352>\004\001\000\312\261\025A\004\001\000\225\025AC\004\001\000`ylE\004\001\000+\335\227G\004\001\000\366@\303I\004\001\000\301\244\356K\004\001\000\214\010\032N\004\001\000WlEP\004\001\000\"\320pR\004\001\000\3553\234T\004\001\000\270\227\307V\004\001\000\203\373\362X\004\001\000N_\036[\004\001\000\031\303I]\004\001\000\344&u_\004\001\000\257\212\240a\004\001\000z\356\313c\004\001\000ER\367e\004\001\000\020\266\"h\004\001\000\333\031Nj\004\001\000\246}yl\004\001\000q\341\244n\004\001\000<E\320p\004\001\000\007\251\373r\004\001\000\322\014\'u\004\001\000\235pRw\004\001\000h\324}y\004\001\00038\251{\004\001\000\376\233\324}"
indirect_data: """"
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 15:51:16.986845 10737 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 15:51:17.187675 10832 tablet_service.cc:1467] Processing CreateTablet for tablet 2edb972a2a164f00a942aa6248187fe8 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260809 15:51:17.188673 10836 tablet_service.cc:1467] Processing CreateTablet for tablet 13ef36e0f4c040a7963771a1dcb2a286 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260809 15:51:17.189141 10832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2edb972a2a164f00a942aa6248187fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.191217 10835 tablet_service.cc:1467] Processing CreateTablet for tablet 1faddf7ecc6e48ba9306d7ece1f936b1 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260809 15:51:17.193419 10834 tablet_service.cc:1467] Processing CreateTablet for tablet 9602fc7c9e814af09c87e4cb17fdb08d (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260809 15:51:17.194495 11008 tablet_service.cc:1467] Processing CreateTablet for tablet 13ef36e0f4c040a7963771a1dcb2a286 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260809 15:51:17.195725 11008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13ef36e0f4c040a7963771a1dcb2a286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.195600 10833 tablet_service.cc:1467] Processing CreateTablet for tablet 5b472f5780ac4f56a2443e8ccc49125f (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260809 15:51:17.199386 11006 tablet_service.cc:1467] Processing CreateTablet for tablet 1faddf7ecc6e48ba9306d7ece1f936b1 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260809 15:51:17.199460 10830 tablet_service.cc:1467] Processing CreateTablet for tablet 8331c9254a504745ac450b9d41ea6ab7 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260809 15:51:17.200634 11006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1faddf7ecc6e48ba9306d7ece1f936b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.201836 10829 tablet_service.cc:1467] Processing CreateTablet for tablet 4ff56f55843f41598bce93ad8056bcb9 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260809 15:51:17.197700 10831 tablet_service.cc:1467] Processing CreateTablet for tablet e80d8e2921024e68a1b7c56fda99ee80 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260809 15:51:17.190008 10836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13ef36e0f4c040a7963771a1dcb2a286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.204382 10835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1faddf7ecc6e48ba9306d7ece1f936b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.204974 10831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e80d8e2921024e68a1b7c56fda99ee80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.206416 10833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b472f5780ac4f56a2443e8ccc49125f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.211104 10829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ff56f55843f41598bce93ad8056bcb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.211805 10834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9602fc7c9e814af09c87e4cb17fdb08d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.211697 11004 tablet_service.cc:1467] Processing CreateTablet for tablet 9602fc7c9e814af09c87e4cb17fdb08d (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260809 15:51:17.212960 11004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9602fc7c9e814af09c87e4cb17fdb08d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.213696 10828 tablet_service.cc:1467] Processing CreateTablet for tablet 5e8d12b5ef9e415cbcb25e62ab228546 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260809 15:51:17.214974 10828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e8d12b5ef9e415cbcb25e62ab228546. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.216064 11003 tablet_service.cc:1467] Processing CreateTablet for tablet 5b472f5780ac4f56a2443e8ccc49125f (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260809 15:51:17.217219 11003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b472f5780ac4f56a2443e8ccc49125f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.219705 10827 tablet_service.cc:1467] Processing CreateTablet for tablet e5757bc9ac9441c3bdfa6c5b2b743b6d (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260809 15:51:17.220947 10827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5757bc9ac9441c3bdfa6c5b2b743b6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.224879 10830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8331c9254a504745ac450b9d41ea6ab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.232689 11002 tablet_service.cc:1467] Processing CreateTablet for tablet 2edb972a2a164f00a942aa6248187fe8 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260809 15:51:17.235179 11001 tablet_service.cc:1467] Processing CreateTablet for tablet e80d8e2921024e68a1b7c56fda99ee80 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260809 15:51:17.237248 11000 tablet_service.cc:1467] Processing CreateTablet for tablet 8331c9254a504745ac450b9d41ea6ab7 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260809 15:51:17.237738 10834 tablet_service.cc:1467] Processing CreateTablet for tablet 73ac44823c384ed99817c1ba3bba75ea (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260809 15:51:17.240052 10999 tablet_service.cc:1467] Processing CreateTablet for tablet 4ff56f55843f41598bce93ad8056bcb9 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260809 15:51:17.233954 11002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2edb972a2a164f00a942aa6248187fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.241595 10834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73ac44823c384ed99817c1ba3bba75ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.243654 11000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8331c9254a504745ac450b9d41ea6ab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.246192 10999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ff56f55843f41598bce93ad8056bcb9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.248769 11001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e80d8e2921024e68a1b7c56fda99ee80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.261790 10996 tablet_service.cc:1467] Processing CreateTablet for tablet e5757bc9ac9441c3bdfa6c5b2b743b6d (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260809 15:51:17.262238 10998 tablet_service.cc:1467] Processing CreateTablet for tablet 5e8d12b5ef9e415cbcb25e62ab228546 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260809 15:51:17.263144 10996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5757bc9ac9441c3bdfa6c5b2b743b6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.266423 10998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e8d12b5ef9e415cbcb25e62ab228546. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.293241 10835 tablet_service.cc:1467] Processing CreateTablet for tablet e6e09d263f96440a99c99d5a5c6ea477 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260809 15:51:17.294674 10835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6e09d263f96440a99c99d5a5c6ea477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.298228 11004 tablet_service.cc:1467] Processing CreateTablet for tablet 73ac44823c384ed99817c1ba3bba75ea (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260809 15:51:17.300962 11065 tablet_bootstrap.cc:492] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:17.316530 11065 tablet_bootstrap.cc:654] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:17.318261 11004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73ac44823c384ed99817c1ba3bba75ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.324033 11003 tablet_service.cc:1467] Processing CreateTablet for tablet e6e09d263f96440a99c99d5a5c6ea477 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260809 15:51:17.325307 11003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6e09d263f96440a99c99d5a5c6ea477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.335063 10999 tablet_service.cc:1467] Processing CreateTablet for tablet f866570ff0c5456ba186b9234cb692c8 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260809 15:51:17.335510 10996 tablet_service.cc:1467] Processing CreateTablet for tablet c695a0d40c7e4b04a1f7a1113954a8e4 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260809 15:51:17.336488 10999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f866570ff0c5456ba186b9234cb692c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.337515 10834 tablet_service.cc:1467] Processing CreateTablet for tablet f866570ff0c5456ba186b9234cb692c8 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260809 15:51:17.339330 10834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f866570ff0c5456ba186b9234cb692c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.344337 10996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c695a0d40c7e4b04a1f7a1113954a8e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.347904 10830 tablet_service.cc:1467] Processing CreateTablet for tablet c695a0d40c7e4b04a1f7a1113954a8e4 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260809 15:51:17.349050 10830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c695a0d40c7e4b04a1f7a1113954a8e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.357515 10827 tablet_service.cc:1467] Processing CreateTablet for tablet 8777c94cbe134bf3aa222691d76ba470 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260809 15:51:17.359362 10832 tablet_service.cc:1467] Processing CreateTablet for tablet 898f053882ab4598a37b00ed268ca6b3 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260809 15:51:17.369905 11067 tablet_bootstrap.cc:492] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:17.375635 11067 tablet_bootstrap.cc:654] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:17.389168 10827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8777c94cbe134bf3aa222691d76ba470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.392944 11001 tablet_service.cc:1467] Processing CreateTablet for tablet 8777c94cbe134bf3aa222691d76ba470 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260809 15:51:17.397506 10829 tablet_service.cc:1467] Processing CreateTablet for tablet e42f46986036495ab1735d596ab06d6d (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260809 15:51:17.399608 11006 tablet_service.cc:1467] Processing CreateTablet for tablet e42f46986036495ab1735d596ab06d6d (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260809 15:51:17.415138 11000 tablet_service.cc:1467] Processing CreateTablet for tablet 898f053882ab4598a37b00ed268ca6b3 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260809 15:51:17.418987 11065 tablet_bootstrap.cc:492] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:17.419456 11065 ts_tablet_manager.cc:1397] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.119s user 0.013s sys 0.030s
I20260809 15:51:17.415910 10999 tablet_service.cc:1467] Processing CreateTablet for tablet 6dbe3aec3ad84a48b982e17d89d8b6ab (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260809 15:51:17.421435 11001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8777c94cbe134bf3aa222691d76ba470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.422039 10919 tablet_service.cc:1467] Processing CreateTablet for tablet 13ef36e0f4c040a7963771a1dcb2a286 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION VALUES < 36398027
I20260809 15:51:17.422299 11065 raft_consensus.cc:348] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:17.423074 11065 raft_consensus.cc:374] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:17.423434 10919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13ef36e0f4c040a7963771a1dcb2a286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.423377 11065 raft_consensus.cc:729] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:17.423601 10831 tablet_service.cc:1467] Processing CreateTablet for tablet 6dbe3aec3ad84a48b982e17d89d8b6ab (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
I20260809 15:51:17.424015 10828 tablet_service.cc:1467] Processing CreateTablet for tablet cfa61e868cfb4915b0572f8e33745f43 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260809 15:51:17.424129 11065 consensus_queue.cc:260] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:17.424801 10831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dbe3aec3ad84a48b982e17d89d8b6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.431329 10917 tablet_service.cc:1467] Processing CreateTablet for tablet 9602fc7c9e814af09c87e4cb17fdb08d (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 72796054 <= VALUES < 109194081
I20260809 15:51:17.431953 10915 tablet_service.cc:1467] Processing CreateTablet for tablet 2edb972a2a164f00a942aa6248187fe8 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 145592108 <= VALUES < 181990135
I20260809 15:51:17.432674 10917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9602fc7c9e814af09c87e4cb17fdb08d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.433247 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2edb972a2a164f00a942aa6248187fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.434671 10916 tablet_service.cc:1467] Processing CreateTablet for tablet 5b472f5780ac4f56a2443e8ccc49125f (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 109194081 <= VALUES < 145592108
I20260809 15:51:17.436035 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b472f5780ac4f56a2443e8ccc49125f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.438711 10832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 898f053882ab4598a37b00ed268ca6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.439822 10914 tablet_service.cc:1467] Processing CreateTablet for tablet e80d8e2921024e68a1b7c56fda99ee80 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 181990135 <= VALUES < 218388162
I20260809 15:51:17.440179 10831 tablet_service.cc:1467] Processing CreateTablet for tablet 049fe6a2ea4f4a15a98b9c6c5755173d (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260809 15:51:17.441021 10914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e80d8e2921024e68a1b7c56fda99ee80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.441375 10831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 049fe6a2ea4f4a15a98b9c6c5755173d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.442456 10918 tablet_service.cc:1467] Processing CreateTablet for tablet 1faddf7ecc6e48ba9306d7ece1f936b1 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 36398027 <= VALUES < 72796054
I20260809 15:51:17.444540 10913 tablet_service.cc:1467] Processing CreateTablet for tablet 8331c9254a504745ac450b9d41ea6ab7 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 218388162 <= VALUES < 254786189
I20260809 15:51:17.447202 10910 tablet_service.cc:1467] Processing CreateTablet for tablet 5e8d12b5ef9e415cbcb25e62ab228546 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 291184216 <= VALUES < 327582243
I20260809 15:51:17.448643 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1faddf7ecc6e48ba9306d7ece1f936b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.451633 10912 tablet_service.cc:1467] Processing CreateTablet for tablet 4ff56f55843f41598bce93ad8056bcb9 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 254786189 <= VALUES < 291184216
I20260809 15:51:17.451757 10909 tablet_service.cc:1467] Processing CreateTablet for tablet e5757bc9ac9441c3bdfa6c5b2b743b6d (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 327582243 <= VALUES < 363980270
I20260809 15:51:17.453609 10913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8331c9254a504745ac450b9d41ea6ab7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.453742 11002 tablet_service.cc:1467] Processing CreateTablet for tablet cfa61e868cfb4915b0572f8e33745f43 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260809 15:51:17.456689 10828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa61e868cfb4915b0572f8e33745f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.462256 11000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 898f053882ab4598a37b00ed268ca6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.473640 10999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dbe3aec3ad84a48b982e17d89d8b6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.473893 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5757bc9ac9441c3bdfa6c5b2b743b6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.474504 11006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42f46986036495ab1735d596ab06d6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.450050 10834 tablet_service.cc:1467] Processing CreateTablet for tablet 06c98b3690e54b19a8bc4b4366fce35a (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260809 15:51:17.487509 11065 ts_tablet_manager.cc:1428] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.068s user 0.013s sys 0.018s
W20260809 15:51:17.496990 10878 tablet.cc:2369] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 15:51:17.494927 10998 tablet_service.cc:1467] Processing CreateTablet for tablet 06c98b3690e54b19a8bc4b4366fce35a (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260809 15:51:17.504685 11067 tablet_bootstrap.cc:492] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:17.505270 11067 ts_tablet_manager.cc:1397] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.136s user 0.017s sys 0.028s
I20260809 15:51:17.508946 11067 raft_consensus.cc:348] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:17.510461 10829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42f46986036495ab1735d596ab06d6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.511070 10912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ff56f55843f41598bce93ad8056bcb9. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 15:51:17.516336 10789 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 15:51:17.516709 10910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e8d12b5ef9e415cbcb25e62ab228546. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.520222 11008 tablet_service.cc:1467] Processing CreateTablet for tablet 9c38f65ac0de428ba1d25c6f0bb6dc91 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260809 15:51:17.528712 11004 tablet_service.cc:1467] Processing CreateTablet for tablet dd8bcd39f37d403f9d8d8f43d870a878 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260809 15:51:17.530696 11002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa61e868cfb4915b0572f8e33745f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.531397 11004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd8bcd39f37d403f9d8d8f43d870a878. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.545075 10835 tablet_service.cc:1467] Processing CreateTablet for tablet 6d4d6dee78914014adab40a391b02955 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260809 15:51:17.546666 10835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d4d6dee78914014adab40a391b02955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.554754 10833 tablet_service.cc:1467] Processing CreateTablet for tablet dd8bcd39f37d403f9d8d8f43d870a878 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260809 15:51:17.555044 10996 tablet_service.cc:1467] Processing CreateTablet for tablet 220f58b662044f089f2be802ed24284a (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260809 15:51:17.555910 10998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c98b3690e54b19a8bc4b4366fce35a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.556113 10833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd8bcd39f37d403f9d8d8f43d870a878. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.558252 10834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c98b3690e54b19a8bc4b4366fce35a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.559454 11008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c38f65ac0de428ba1d25c6f0bb6dc91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.480844 10836 tablet_service.cc:1467] Processing CreateTablet for tablet 9c38f65ac0de428ba1d25c6f0bb6dc91 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260809 15:51:17.560551 10836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c38f65ac0de428ba1d25c6f0bb6dc91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.562736 11003 tablet_service.cc:1467] Processing CreateTablet for tablet 049fe6a2ea4f4a15a98b9c6c5755173d (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260809 15:51:17.509931 11067 raft_consensus.cc:374] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:17.595189 11072 tablet_bootstrap.cc:492] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:17.575102 11067 raft_consensus.cc:729] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:17.574926 11001 tablet_service.cc:1467] Processing CreateTablet for tablet 6d4d6dee78914014adab40a391b02955 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260809 15:51:17.603070 11000 tablet_service.cc:1467] Processing CreateTablet for tablet 11cab658217a44bebba11963c723e3dc (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260809 15:51:17.606240 11004 tablet_service.cc:1467] Processing CreateTablet for tablet b6d399a7394c4a59b90a308aedfdf899 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260809 15:51:17.619186 11002 tablet_service.cc:1467] Processing CreateTablet for tablet 81af4312dd044dfa85b9067b2fe4974c (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260809 15:51:17.610389 11008 tablet_service.cc:1467] Processing CreateTablet for tablet 974451500a6e434f8d407d9f178ab710 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260809 15:51:17.615588 10918 tablet_service.cc:1467] Processing CreateTablet for tablet f866570ff0c5456ba186b9234cb692c8 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 436776324 <= VALUES < 473174351
I20260809 15:51:17.621459 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f866570ff0c5456ba186b9234cb692c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.596315 10996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 220f58b662044f089f2be802ed24284a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.623219 11001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d4d6dee78914014adab40a391b02955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.626420 11000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11cab658217a44bebba11963c723e3dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.630389 11003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 049fe6a2ea4f4a15a98b9c6c5755173d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.632258 11072 tablet_bootstrap.cc:654] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:17.617089 11067 consensus_queue.cc:260] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:17.634886 11004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6d399a7394c4a59b90a308aedfdf899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.640307 10999 tablet_service.cc:1467] Processing CreateTablet for tablet 2b4306bfa5464b0dbf8301756455ba50 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260809 15:51:17.648568 11001 tablet_service.cc:1467] Processing CreateTablet for tablet fe21e0688ade405bb4590b9c256bd696 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260809 15:51:17.641947 10999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b4306bfa5464b0dbf8301756455ba50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.674746 11001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe21e0688ade405bb4590b9c256bd696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.645548 11006 tablet_service.cc:1467] Processing CreateTablet for tablet 908409e5747b470cabbc8aed443397f7 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260809 15:51:17.677160 11006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 908409e5747b470cabbc8aed443397f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.678709 11008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 974451500a6e434f8d407d9f178ab710. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.679837 11002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81af4312dd044dfa85b9067b2fe4974c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.595803 10915 tablet_service.cc:1467] Processing CreateTablet for tablet e6e09d263f96440a99c99d5a5c6ea477 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 400378297 <= VALUES < 436776324
I20260809 15:51:17.684212 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6e09d263f96440a99c99d5a5c6ea477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.622067 10919 tablet_service.cc:1467] Processing CreateTablet for tablet c695a0d40c7e4b04a1f7a1113954a8e4 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 473174351 <= VALUES < 509572378
I20260809 15:51:17.689960 10919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c695a0d40c7e4b04a1f7a1113954a8e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.648779 11048 heartbeater.cc:503] Master 127.9.84.62:42773 was elected leader, sending a full tablet report...
I20260809 15:51:17.699018 10877 heartbeater.cc:503] Master 127.9.84.62:42773 was elected leader, sending a full tablet report...
I20260809 15:51:17.585500 10913 tablet_service.cc:1467] Processing CreateTablet for tablet 73ac44823c384ed99817c1ba3bba75ea (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 363980270 <= VALUES < 400378297
I20260809 15:51:17.649075 11067 ts_tablet_manager.cc:1428] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.143s user 0.005s sys 0.005s
I20260809 15:51:17.715162 11067 tablet_bootstrap.cc:492] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:17.722491 11070 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:17.723271 11070 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:17.733793 11065 tablet_bootstrap.cc:492] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:17.735854 10832 tablet_service.cc:1467] Processing CreateTablet for tablet 220f58b662044f089f2be802ed24284a (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260809 15:51:17.740128 11070 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:17.748668 10830 tablet_service.cc:1467] Processing CreateTablet for tablet b6d399a7394c4a59b90a308aedfdf899 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260809 15:51:17.749226 10831 tablet_service.cc:1467] Processing CreateTablet for tablet 11cab658217a44bebba11963c723e3dc (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260809 15:51:17.766781 11002 tablet_service.cc:1467] Processing CreateTablet for tablet 9ff6a4ed964b47dba2d0ea0a114e97f4 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260809 15:51:17.768011 11002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ff6a4ed964b47dba2d0ea0a114e97f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.768750 10913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73ac44823c384ed99817c1ba3bba75ea. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 15:51:17.772707 10719 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 102793us
I20260809 15:51:17.793708 11073 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:17.794225 11073 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:17.796707 11073 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:17.799966 11072 tablet_bootstrap.cc:492] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:17.800726 11072 ts_tablet_manager.cc:1397] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.206s user 0.010s sys 0.002s
I20260809 15:51:17.811480 10916 tablet_service.cc:1467] Processing CreateTablet for tablet 898f053882ab4598a37b00ed268ca6b3 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 545970405 <= VALUES < 582368432
I20260809 15:51:17.813114 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 898f053882ab4598a37b00ed268ca6b3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.821475 10914 tablet_service.cc:1467] Processing CreateTablet for tablet 6dbe3aec3ad84a48b982e17d89d8b6ab (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 618766459 <= VALUES < 655164486
W20260809 15:51:17.835112 10722 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 108465us
I20260809 15:51:17.837464 11072 raft_consensus.cc:348] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:17.845602 11072 raft_consensus.cc:374] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:17.855274 11072 raft_consensus.cc:729] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:17.850973 10831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11cab658217a44bebba11963c723e3dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.862457 10832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 220f58b662044f089f2be802ed24284a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.862694 10960 heartbeater.cc:503] Master 127.9.84.62:42773 was elected leader, sending a full tablet report...
I20260809 15:51:17.870689 11002 tablet_service.cc:1467] Processing CreateTablet for tablet b6ba8250f4ef43bfbf8f517e86a131a1 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260809 15:51:17.871929 11002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6ba8250f4ef43bfbf8f517e86a131a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.874598 10830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6d399a7394c4a59b90a308aedfdf899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.883178 10909 tablet_service.cc:1467] Processing CreateTablet for tablet 049fe6a2ea4f4a15a98b9c6c5755173d (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 691562513 <= VALUES < 727960540
I20260809 15:51:17.883968 11067 tablet_bootstrap.cc:654] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:17.884500 11065 tablet_bootstrap.cc:654] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:17.861454 10915 tablet_service.cc:1467] Processing CreateTablet for tablet cfa61e868cfb4915b0572f8e33745f43 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 655164486 <= VALUES < 691562513
I20260809 15:51:17.886227 10918 tablet_service.cc:1467] Processing CreateTablet for tablet 06c98b3690e54b19a8bc4b4366fce35a (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 727960540 <= VALUES < 764358567
I20260809 15:51:17.889633 11008 tablet_service.cc:1467] Processing CreateTablet for tablet 9a5ecb5bb6e347a6921a7c894853e92e (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260809 15:51:17.891379 10998 tablet_service.cc:1467] Processing CreateTablet for tablet 08f368b4f4e748c596049c81eaed3f70 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260809 15:51:17.901659 10998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f368b4f4e748c596049c81eaed3f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.905032 10996 tablet_service.cc:1467] Processing CreateTablet for tablet 855d87c571d0427ba84900052dd9e2c0 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260809 15:51:17.856266 11003 tablet_service.cc:1467] Processing CreateTablet for tablet 3d87bec6301b44c7b8dd46364e6110a8 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
W20260809 15:51:17.911135 10722 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 55258.7us
I20260809 15:51:17.919433 10996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 855d87c571d0427ba84900052dd9e2c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.930612 10917 tablet_service.cc:1467] Processing CreateTablet for tablet e42f46986036495ab1735d596ab06d6d (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 582368432 <= VALUES < 618766459
I20260809 15:51:17.931907 10914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6dbe3aec3ad84a48b982e17d89d8b6ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.932577 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa61e868cfb4915b0572f8e33745f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.936092 10912 tablet_service.cc:1467] Processing CreateTablet for tablet 9c38f65ac0de428ba1d25c6f0bb6dc91 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 764358567 <= VALUES < 800756594
I20260809 15:51:17.937568 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 049fe6a2ea4f4a15a98b9c6c5755173d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.943562 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c98b3690e54b19a8bc4b4366fce35a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.948258 11008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a5ecb5bb6e347a6921a7c894853e92e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.954414 11003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d87bec6301b44c7b8dd46364e6110a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.953711 10835 tablet_service.cc:1467] Processing CreateTablet for tablet 2b4306bfa5464b0dbf8301756455ba50 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260809 15:51:17.955504 10835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b4306bfa5464b0dbf8301756455ba50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.958189 10836 tablet_service.cc:1467] Processing CreateTablet for tablet 908409e5747b470cabbc8aed443397f7 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260809 15:51:17.959579 10836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 908409e5747b470cabbc8aed443397f7. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 15:51:17.906706 10719 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 55715.5us
I20260809 15:51:17.908959 11072 consensus_queue.cc:260] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:17.972518 11004 tablet_service.cc:1467] Processing CreateTablet for tablet 7c8eaf50bcb3457bb4b1e30406cc349f (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260809 15:51:17.974828 10829 tablet_service.cc:1467] Processing CreateTablet for tablet fe21e0688ade405bb4590b9c256bd696 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260809 15:51:18.112705 10999 tablet_service.cc:1467] Processing CreateTablet for tablet 0b9fb9f88a6847af87fef710d1a2793b (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260809 15:51:18.113135 11004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c8eaf50bcb3457bb4b1e30406cc349f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.113216 10829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe21e0688ade405bb4590b9c256bd696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.113894 10999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b9fb9f88a6847af87fef710d1a2793b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:17.997143 11072 ts_tablet_manager.cc:1428] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.196s user 0.013s sys 0.010s
I20260809 15:51:17.994921 11065 tablet_bootstrap.cc:492] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:18.119920 11065 ts_tablet_manager.cc:1397] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.387s user 0.005s sys 0.017s
I20260809 15:51:18.003773 11067 tablet_bootstrap.cc:492] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:18.201699 11067 ts_tablet_manager.cc:1397] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.487s user 0.022s sys 0.025s
I20260809 15:51:18.011652 10835 tablet_service.cc:1467] Processing CreateTablet for tablet 9ff6a4ed964b47dba2d0ea0a114e97f4 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260809 15:51:18.202934 10835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ff6a4ed964b47dba2d0ea0a114e97f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.204277 10998 tablet_service.cc:1467] Processing CreateTablet for tablet f4d1cc07da5b49ae95998fabc2323d24 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260809 15:51:18.205586 10998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4d1cc07da5b49ae95998fabc2323d24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.013345 10827 tablet_service.cc:1467] Processing CreateTablet for tablet 974451500a6e434f8d407d9f178ab710 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260809 15:51:18.017365 10836 tablet_service.cc:1467] Processing CreateTablet for tablet b6ba8250f4ef43bfbf8f517e86a131a1 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260809 15:51:18.070220 10831 tablet_service.cc:1467] Processing CreateTablet for tablet 3d87bec6301b44c7b8dd46364e6110a8 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260809 15:51:18.206641 10999 tablet_service.cc:1467] Processing CreateTablet for tablet 0f891fe62df749108fa553cb0661a874 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260809 15:51:18.078730 10909 tablet_service.cc:1467] Processing CreateTablet for tablet dd8bcd39f37d403f9d8d8f43d870a878 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 800756594 <= VALUES < 837154621
I20260809 15:51:18.079684 10917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42f46986036495ab1735d596ab06d6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.209141 10912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c38f65ac0de428ba1d25c6f0bb6dc91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.209075 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:18.081794 10910 tablet_service.cc:1467] Processing CreateTablet for tablet 8777c94cbe134bf3aa222691d76ba470 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 509572378 <= VALUES < 545970405
I20260809 15:51:18.209844 10848 raft_consensus.cc:2454] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:18.210711 10910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8777c94cbe134bf3aa222691d76ba470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.211288 10978 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:18.212062 11073 raft_consensus.cc:2781] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:18.212423 11073 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:18.212750 11073 raft_consensus.cc:3037] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:18.086884 10918 tablet_service.cc:1467] Processing CreateTablet for tablet 6d4d6dee78914014adab40a391b02955 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 837154621 <= VALUES < 873552648
I20260809 15:51:18.214810 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d4d6dee78914014adab40a391b02955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.215983 11003 tablet_service.cc:1467] Processing CreateTablet for tablet 02f25d93293a40729fd751a19ab9adeb (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260809 15:51:18.088743 10919 tablet_service.cc:1467] Processing CreateTablet for tablet 220f58b662044f089f2be802ed24284a (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 873552648 <= VALUES < 909950675
I20260809 15:51:18.218312 10832 tablet_service.cc:1467] Processing CreateTablet for tablet 0b9fb9f88a6847af87fef710d1a2793b (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260809 15:51:18.218932 10919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 220f58b662044f089f2be802ed24284a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.220007 10996 tablet_service.cc:1467] Processing CreateTablet for tablet 9d3f35b0ae8e4b02b72f832ca1518464 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260809 15:51:18.221513 10915 tablet_service.cc:1467] Processing CreateTablet for tablet 974451500a6e434f8d407d9f178ab710 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 982746729 <= VALUES < 1019144756
I20260809 15:51:17.895025 10833 tablet_service.cc:1467] Processing CreateTablet for tablet 81af4312dd044dfa85b9067b2fe4974c (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260809 15:51:18.117787 11072 tablet_bootstrap.cc:492] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:18.123124 10916 tablet_service.cc:1467] Processing CreateTablet for tablet 11cab658217a44bebba11963c723e3dc (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 909950675 <= VALUES < 946348702
I20260809 15:51:18.222738 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 974451500a6e434f8d407d9f178ab710. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.223435 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11cab658217a44bebba11963c723e3dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.225121 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd8bcd39f37d403f9d8d8f43d870a878. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.125521 11065 raft_consensus.cc:348] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.226253 11065 raft_consensus.cc:374] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:18.226632 11065 raft_consensus.cc:729] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:18.227447 11065 consensus_queue.cc:260] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.135492 10829 tablet_service.cc:1467] Processing CreateTablet for tablet 08f368b4f4e748c596049c81eaed3f70 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260809 15:51:18.137596 11006 tablet_service.cc:1467] Processing CreateTablet for tablet 8975b3e7ddc544bc888dd61fffc5313b (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260809 15:51:18.144793 11000 tablet_service.cc:1467] Processing CreateTablet for tablet 5dc95cd5fbb945e7af0f89cf2ec436c9 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260809 15:51:18.154994 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:18.176015 11001 tablet_service.cc:1467] Processing CreateTablet for tablet 87c018b6b9eb45cab9c8349a86ffb7d2 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260809 15:51:18.169723 10830 tablet_service.cc:1467] Processing CreateTablet for tablet 855d87c571d0427ba84900052dd9e2c0 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260809 15:51:18.179898 10834 tablet_service.cc:1467] Processing CreateTablet for tablet 7c8eaf50bcb3457bb4b1e30406cc349f (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260809 15:51:18.185416 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:18.237150 11067 raft_consensus.cc:348] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.195086 10913 tablet_service.cc:1467] Processing CreateTablet for tablet b6d399a7394c4a59b90a308aedfdf899 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 946348702 <= VALUES < 982746729
I20260809 15:51:18.195950 11083 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:18.238503 11067 raft_consensus.cc:374] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:18.198022 11008 tablet_service.cc:1467] Processing CreateTablet for tablet 93303d409ab940409d87013868417907 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260809 15:51:18.196640 11004 tablet_service.cc:1467] Processing CreateTablet for tablet 0b2b0df042e44725a7c91deae3df1956 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260809 15:51:18.088749 10828 tablet_service.cc:1467] Processing CreateTablet for tablet 9a5ecb5bb6e347a6921a7c894853e92e (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260809 15:51:18.270226 11067 raft_consensus.cc:729] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:18.188030 11002 tablet_service.cc:1467] Processing CreateTablet for tablet 6a89bcd329414d49a3bfdf5ac68523e9 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260809 15:51:18.206812 10827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 974451500a6e434f8d407d9f178ab710. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.271258 11067 consensus_queue.cc:260] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.238401 11083 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.272341 10829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f368b4f4e748c596049c81eaed3f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.196722 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:18.275072 10931 raft_consensus.cc:2454] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:18.239743 10913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6d399a7394c4a59b90a308aedfdf899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.278815 10831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d87bec6301b44c7b8dd46364e6110a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.279454 10833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81af4312dd044dfa85b9067b2fe4974c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.280023 10836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6ba8250f4ef43bfbf8f517e86a131a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.281512 10834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c8eaf50bcb3457bb4b1e30406cc349f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.283242 10828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a5ecb5bb6e347a6921a7c894853e92e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.287214 11083 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:18.241582 11072 tablet_bootstrap.cc:654] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:18.242789 11065 ts_tablet_manager.cc:1428] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.123s user 0.005s sys 0.002s
I20260809 15:51:18.252351 11073 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.263959 10999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f891fe62df749108fa553cb0661a874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.295682 11065 tablet_bootstrap.cc:492] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:18.315905 10832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b9fb9f88a6847af87fef710d1a2793b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.322394 11002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a89bcd329414d49a3bfdf5ac68523e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.326355 11018 raft_consensus.cc:2379] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 1: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:51:18.329157 10809 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:18.330236 11070 raft_consensus.cc:3037] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:18.331557 11073 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:18.332255 11073 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:18.332640 11073 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.326361 11000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dc95cd5fbb945e7af0f89cf2ec436c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.336491 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:18.341136 10912 tablet_service.cc:1467] Processing CreateTablet for tablet 2b4306bfa5464b0dbf8301756455ba50 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1055542783 <= VALUES < 1091940810
I20260809 15:51:18.330181 10910 tablet_service.cc:1467] Processing CreateTablet for tablet 81af4312dd044dfa85b9067b2fe4974c (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1019144756 <= VALUES < 1055542783
I20260809 15:51:18.362236 10830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 855d87c571d0427ba84900052dd9e2c0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.367473 10912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b4306bfa5464b0dbf8301756455ba50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.367389 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:18.368219 11001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87c018b6b9eb45cab9c8349a86ffb7d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.368109 10847 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:18.368746 10910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81af4312dd044dfa85b9067b2fe4974c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.369378 10978 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:18.370800 10835 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 32) took 1082 ms. Trace:
I20260809 15:51:18.371622 11073 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:18.372254 11073 raft_consensus.cc:2781] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:18.372637 11073 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:18.373013 11073 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:18.375927 10917 tablet_service.cc:1467] Processing CreateTablet for tablet 908409e5747b470cabbc8aed443397f7 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1091940810 <= VALUES < 1128338837
I20260809 15:51:18.377121 10917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 908409e5747b470cabbc8aed443397f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.391251 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:18.392010 10931 raft_consensus.cc:3037] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:18.394033 11003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02f25d93293a40729fd751a19ab9adeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.394534 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:18.395725 11004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b2b0df042e44725a7c91deae3df1956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.371261 10835 rpcz_store.cc:276] 0809 15:51:17.288678 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.010689 (+722011us) service_pool.cc:227] Handling call
0809 15:51:18.202446 (+191757us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.202549 (+ 103us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.275193 (+ 72644us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.336177 (+ 60984us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.370705 (+ 34528us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.402486 11008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93303d409ab940409d87013868417907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.406373 11006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8975b3e7ddc544bc888dd61fffc5313b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.408619 11070 raft_consensus.cc:2726] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 15:51:18.414525 10978 leader_election.cc:341] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:18.420092 11065 tablet_bootstrap.cc:654] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:18.422461 10996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d3f35b0ae8e4b02b72f832ca1518464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.423269 10913 tablet_service.cc:1467] Processing CreateTablet for tablet 9ff6a4ed964b47dba2d0ea0a114e97f4 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1164736864 <= VALUES < 1201134891
I20260809 15:51:18.424539 10913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ff6a4ed964b47dba2d0ea0a114e97f4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.417881 10848 raft_consensus.cc:2454] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:18.397825 10835 tablet_service.cc:1467] Processing CreateTablet for tablet 8975b3e7ddc544bc888dd61fffc5313b (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260809 15:51:18.454653 10835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8975b3e7ddc544bc888dd61fffc5313b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.456707 10914 tablet_service.cc:1467] Processing CreateTablet for tablet b6ba8250f4ef43bfbf8f517e86a131a1 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1201134891 <= VALUES < 1237532918
I20260809 15:51:18.461716 10919 tablet_service.cc:1467] Processing CreateTablet for tablet 3d87bec6301b44c7b8dd46364e6110a8 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1237532918 <= VALUES < 1273930945
I20260809 15:51:18.471711 10931 raft_consensus.cc:2454] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:18.472888 10978 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:18.473771 11087 raft_consensus.cc:2781] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:18.474268 11087 raft_consensus.cc:686] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:18.458030 10914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6ba8250f4ef43bfbf8f517e86a131a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.475101 11087 consensus_queue.cc:237] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.476706 11002 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50526 (request call id 43) took 1071 ms. Trace:
I20260809 15:51:18.393560 10916 tablet_service.cc:1467] Processing CreateTablet for tablet fe21e0688ade405bb4590b9c256bd696 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1128338837 <= VALUES < 1164736864
I20260809 15:51:18.482009 11002 rpcz_store.cc:276] 0809 15:51:17.405754 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.185167 (+779413us) service_pool.cc:227] Handling call
0809 15:51:18.271258 (+ 86091us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.271352 (+ 94us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.414896 (+143544us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.476237 (+ 61341us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.476649 (+ 412us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.485307 10832 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 39) took 1177 ms. Trace:
I20260809 15:51:18.485596 10833 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 28) took 1198 ms. Trace:
I20260809 15:51:18.486181 10833 rpcz_store.cc:276] 0809 15:51:17.287416 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:17.856203 (+568787us) service_pool.cc:227] Handling call
0809 15:51:18.222344 (+366141us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.222420 (+ 76us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.352984 (+130564us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.448499 (+ 95515us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.485540 (+ 37041us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4919}
I20260809 15:51:18.487488 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:18.488199 10834 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 38) took 1180 ms. Trace:
I20260809 15:51:18.488584 10834 rpcz_store.cc:276] 0809 15:51:17.307693 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.178954 (+871261us) service_pool.cc:227] Handling call
0809 15:51:18.232167 (+ 53213us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.232262 (+ 95us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.382128 (+149866us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.487742 (+105614us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.488138 (+ 396us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.489292 10833 tablet_service.cc:1467] Processing CreateTablet for tablet 5dc95cd5fbb945e7af0f89cf2ec436c9 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260809 15:51:18.491230 10833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dc95cd5fbb945e7af0f89cf2ec436c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.491484 11067 ts_tablet_manager.cc:1428] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.289s user 0.012s sys 0.015s
I20260809 15:51:18.485713 10832 rpcz_store.cc:276] 0809 15:51:17.308039 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.214009 (+905970us) service_pool.cc:227] Handling call
0809 15:51:18.219112 (+ 5103us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.219203 (+ 91us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.363989 (+144786us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.473363 (+109374us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.485249 (+ 11886us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":6343,"spinlock_wait_cycles":905472}
I20260809 15:51:18.496793 10832 tablet_service.cc:1467] Processing CreateTablet for tablet 6a89bcd329414d49a3bfdf5ac68523e9 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260809 15:51:18.498446 11067 tablet_bootstrap.cc:492] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:18.495038 10919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d87bec6301b44c7b8dd46364e6110a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.500262 10910 tablet_service.cc:1467] Processing CreateTablet for tablet 08f368b4f4e748c596049c81eaed3f70 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1310328972 <= VALUES < 1346726999
I20260809 15:51:18.494884 10834 tablet_service.cc:1467] Processing CreateTablet for tablet 87c018b6b9eb45cab9c8349a86ffb7d2 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260809 15:51:18.512015 10834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87c018b6b9eb45cab9c8349a86ffb7d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.513227 11072 tablet_bootstrap.cc:492] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:18.513696 11072 ts_tablet_manager.cc:1397] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.396s user 0.027s sys 0.027s
I20260809 15:51:18.520503 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe21e0688ade405bb4590b9c256bd696. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.523118 11072 raft_consensus.cc:348] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.523875 11072 raft_consensus.cc:374] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:18.524142 11072 raft_consensus.cc:729] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:18.524878 11072 consensus_queue.cc:260] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.531867 10910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f368b4f4e748c596049c81eaed3f70. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 15:51:18.535833 10789 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 15:51:18.532003 10832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a89bcd329414d49a3bfdf5ac68523e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.532226 10828 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 35) took 1224 ms. Trace:
I20260809 15:51:18.506987 11073 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.541903 10828 rpcz_store.cc:276] 0809 15:51:17.306797 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.087860 (+781063us) service_pool.cc:227] Handling call
0809 15:51:18.270656 (+182796us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.270760 (+ 104us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.352284 (+ 81524us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.493219 (+140935us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.531251 (+ 38032us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4729}
I20260809 15:51:18.546432 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
W20260809 15:51:18.548066 10978 leader_election.cc:341] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:18.549082 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:18.549876 10848 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:18.551821 10828 tablet_service.cc:1467] Processing CreateTablet for tablet 0b2b0df042e44725a7c91deae3df1956 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260809 15:51:18.553290 10828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b2b0df042e44725a7c91deae3df1956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.555202 11073 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:18.507473 11002 tablet_service.cc:1467] Processing CreateTablet for tablet 369529a0023c4cf4a709bb1a480052b4 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260809 15:51:18.560854 11002 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 369529a0023c4cf4a709bb1a480052b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.566722 10998 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50526 (request call id 46) took 1039 ms. Trace:
I20260809 15:51:18.567625 10831 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 34) took 1261 ms. Trace:
I20260809 15:51:18.568141 10831 rpcz_store.cc:276] 0809 15:51:17.306453 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.069473 (+763020us) service_pool.cc:227] Handling call
0809 15:51:18.207758 (+138285us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.207847 (+ 89us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.425096 (+217249us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.531687 (+106591us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.567555 (+ 35868us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.567245 10998 rpcz_store.cc:276] 0809 15:51:17.563043 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.203347 (+640304us) service_pool.cc:227] Handling call
0809 15:51:18.205120 (+ 1773us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.205225 (+ 105us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.454824 (+249599us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.519243 (+ 64419us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.566646 (+ 47403us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.573019 10998 tablet_service.cc:1467] Processing CreateTablet for tablet 34001ced69f34c6f930f5218eef46524 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260809 15:51:18.574285 10998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34001ced69f34c6f930f5218eef46524. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.584285 11072 ts_tablet_manager.cc:1428] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.070s user 0.007s sys 0.000s
I20260809 15:51:18.578286 10999 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50526 (request call id 47) took 1014 ms. Trace:
I20260809 15:51:18.590929 11072 tablet_bootstrap.cc:492] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:18.580753 10827 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 27) took 1293 ms. Trace:
I20260809 15:51:18.591560 10827 rpcz_store.cc:276] 0809 15:51:17.287120 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:17.829573 (+542453us) service_pool.cc:227] Handling call
0809 15:51:18.206358 (+376785us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.206467 (+ 109us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.405156 (+198689us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.580258 (+175102us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.580700 (+ 442us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.587721 10848 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:18.590451 10999 rpcz_store.cc:276] 0809 15:51:17.610378 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.205015 (+594637us) service_pool.cc:227] Handling call
0809 15:51:18.208221 (+ 3206us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.208452 (+ 231us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.376713 (+168261us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.574760 (+198047us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.578228 (+ 3468us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.593428 10827 tablet_service.cc:1467] Processing CreateTablet for tablet f4d1cc07da5b49ae95998fabc2323d24 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260809 15:51:18.594656 10999 tablet_service.cc:1467] Processing CreateTablet for tablet bf6504ce1c0d4241bfb97eb553c8df35 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260809 15:51:18.595326 10918 tablet_service.cc:1467] Processing CreateTablet for tablet 9a5ecb5bb6e347a6921a7c894853e92e (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1273930945 <= VALUES < 1310328972
I20260809 15:51:18.595952 10999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6504ce1c0d4241bfb97eb553c8df35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.586491 11003 tablet_service.cc:1467] Processing CreateTablet for tablet 279308d651cf45ff86edaae4483bcb79 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260809 15:51:18.597481 11003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 279308d651cf45ff86edaae4483bcb79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.600276 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a5ecb5bb6e347a6921a7c894853e92e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.602070 10996 tablet_service.cc:1467] Processing CreateTablet for tablet 241f07ed14b041cdb37e8ec792283d52 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260809 15:51:18.602523 10829 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 36) took 1295 ms. Trace:
I20260809 15:51:18.602968 10829 rpcz_store.cc:276] 0809 15:51:17.307103 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.134491 (+827388us) service_pool.cc:227] Handling call
0809 15:51:18.229666 (+ 95175us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.229771 (+ 105us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.471274 (+241503us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.601966 (+130692us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.602468 (+ 502us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.603574 11000 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50526 (request call id 41) took 1198 ms. Trace:
I20260809 15:51:18.603721 11001 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50526 (request call id 42) took 1198 ms. Trace:
I20260809 15:51:18.604070 11000 rpcz_store.cc:276] 0809 15:51:17.405148 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.143609 (+738461us) service_pool.cc:227] Handling call
0809 15:51:18.230792 (+ 87183us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.230879 (+ 87us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.401032 (+170153us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.603119 (+202087us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.603522 (+ 403us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":7988}
I20260809 15:51:18.594687 10827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4d1cc07da5b49ae95998fabc2323d24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.607007 10978 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:18.608140 11073 raft_consensus.cc:2781] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:18.608621 11073 raft_consensus.cc:686] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:18.609488 11073 consensus_queue.cc:237] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.578071 10912 tablet_service.cc:1467] Processing CreateTablet for tablet 855d87c571d0427ba84900052dd9e2c0 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1346726999 <= VALUES < 1383125026
I20260809 15:51:18.615289 10912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 855d87c571d0427ba84900052dd9e2c0. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 15:51:18.618270 10722 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 74865.1us
I20260809 15:51:18.620733 10831 tablet_service.cc:1467] Processing CreateTablet for tablet 93303d409ab940409d87013868417907 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260809 15:51:18.622092 10831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93303d409ab940409d87013868417907. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 15:51:18.622437 10719 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.CreateTablet -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 49992.3us
I20260809 15:51:18.578837 11067 tablet_bootstrap.cc:654] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:18.604156 11001 rpcz_store.cc:276] 0809 15:51:17.405443 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.175048 (+769605us) service_pool.cc:227] Handling call
0809 15:51:18.231394 (+ 56346us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.231484 (+ 90us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.482443 (+250959us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.603249 (+120806us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.603671 (+ 422us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5194}
I20260809 15:51:18.639567 10836 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 33) took 1350 ms. Trace:
I20260809 15:51:18.640004 10836 rpcz_store.cc:276] 0809 15:51:17.288965 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.016321 (+727356us) service_pool.cc:227] Handling call
0809 15:51:18.207246 (+190925us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.207329 (+ 83us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.353174 (+145845us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.639116 (+285942us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.639512 (+ 396us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.641644 10835 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 40) took 1333 ms. Trace:
I20260809 15:51:18.642587 10996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 241f07ed14b041cdb37e8ec792283d52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.642241 11000 tablet_service.cc:1467] Processing CreateTablet for tablet 70c84efc43004b7b8ef2f9b869250509 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260809 15:51:18.643363 10909 tablet_service.cc:1467] Processing CreateTablet for tablet 0b9fb9f88a6847af87fef710d1a2793b (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1419523053 <= VALUES < 1455921080
I20260809 15:51:18.644662 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b9fb9f88a6847af87fef710d1a2793b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.652877 10830 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 37) took 1345 ms. Trace:
I20260809 15:51:18.653309 10830 rpcz_store.cc:276] 0809 15:51:17.307396 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.168671 (+861275us) service_pool.cc:227] Handling call
0809 15:51:18.231586 (+ 62915us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.231699 (+ 113us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.540588 (+308889us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.652407 (+111819us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.652822 (+ 415us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.662606 11000 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70c84efc43004b7b8ef2f9b869250509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.652140 10835 rpcz_store.cc:276] 0809 15:51:17.308337 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.396640 (+1088303us) service_pool.cc:227] Handling call
0809 15:51:18.454203 (+ 57563us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.454302 (+ 99us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.518486 (+ 64184us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.641164 (+122678us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.641586 (+ 422us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.664780 11006 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50526 (request call id 40) took 1260 ms. Trace:
I20260809 15:51:18.665205 11006 rpcz_store.cc:276] 0809 15:51:17.404803 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.136568 (+731765us) service_pool.cc:227] Handling call
0809 15:51:18.230235 (+ 93667us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.230330 (+ 95us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.507704 (+277374us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.650187 (+142483us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.664722 (+ 14535us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":26494}
I20260809 15:51:18.666356 11001 tablet_service.cc:1467] Processing CreateTablet for tablet 8feeb57bf5cf4fd1bb17ab7f575222d0 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260809 15:51:18.667693 11001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8feeb57bf5cf4fd1bb17ab7f575222d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.667368 11006 tablet_service.cc:1467] Processing CreateTablet for tablet 6508f17228904310be3c95818973f3a1 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260809 15:51:18.625880 11008 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50526 (request call id 45) took 1169 ms. Trace:
I20260809 15:51:18.668606 11006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6508f17228904310be3c95818973f3a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.664789 10835 tablet_service.cc:1467] Processing CreateTablet for tablet 02f25d93293a40729fd751a19ab9adeb (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260809 15:51:18.682235 10737 catalog_manager.cc:5495] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:18.684018 10737 catalog_manager.cc:5495] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:18.670651 10829 tablet_service.cc:1467] Processing CreateTablet for tablet 0f891fe62df749108fa553cb0661a874 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260809 15:51:18.683184 10835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02f25d93293a40729fd751a19ab9adeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.684377 10915 tablet_service.cc:1467] Processing CreateTablet for tablet 5dc95cd5fbb945e7af0f89cf2ec436c9 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1492319107 <= VALUES < 1528717134
I20260809 15:51:18.688194 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dc95cd5fbb945e7af0f89cf2ec436c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.690943 10830 tablet_service.cc:1467] Processing CreateTablet for tablet 9d3f35b0ae8e4b02b72f832ca1518464 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260809 15:51:18.692327 10830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d3f35b0ae8e4b02b72f832ca1518464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.702484 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:18.703277 10848 raft_consensus.cc:2379] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 1: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:51:18.704591 10893 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:18.705788 11083 raft_consensus.cc:2726] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:18.706621 11072 tablet_bootstrap.cc:654] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:18.733645 10829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f891fe62df749108fa553cb0661a874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.739239 10998 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50526 (request call id 51) took 1127 ms. Trace:
I20260809 15:51:18.739655 10998 rpcz_store.cc:276] 0809 15:51:17.611848 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.572109 (+960261us) service_pool.cc:227] Handling call
0809 15:51:18.573813 (+ 1704us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.573920 (+ 107us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.654304 (+ 80384us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.738731 (+ 84427us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.739182 (+ 451us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.741576 10998 tablet_service.cc:1467] Processing CreateTablet for tablet 1f9b7d9e57934596b377ee8dc79ea23e (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260809 15:51:18.743768 10913 tablet_service.cc:1467] Processing CreateTablet for tablet 87c018b6b9eb45cab9c8349a86ffb7d2 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1528717134 <= VALUES < 1565115161
I20260809 15:51:18.745003 10913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87c018b6b9eb45cab9c8349a86ffb7d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.722731 11065 tablet_bootstrap.cc:492] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:18.714766 11004 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50526 (request call id 44) took 1308 ms. Trace:
I20260809 15:51:18.716471 10836 tablet_service.cc:1467] Processing CreateTablet for tablet 8feeb57bf5cf4fd1bb17ab7f575222d0 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260809 15:51:18.748127 11004 rpcz_store.cc:276] 0809 15:51:17.406040 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.193313 (+787273us) service_pool.cc:227] Handling call
0809 15:51:18.270920 (+ 77607us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.271132 (+ 212us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.588234 (+317102us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.714235 (+126001us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.714703 (+ 468us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.749316 11065 ts_tablet_manager.cc:1397] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.454s user 0.016s sys 0.014s
I20260809 15:51:18.748093 10914 tablet_service.cc:1467] Processing CreateTablet for tablet 6a89bcd329414d49a3bfdf5ac68523e9 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1565115161 <= VALUES < 1601513188
I20260809 15:51:18.750602 10916 tablet_service.cc:1467] Processing CreateTablet for tablet 7c8eaf50bcb3457bb4b1e30406cc349f (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1383125026 <= VALUES < 1419523053
I20260809 15:51:18.751127 10914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a89bcd329414d49a3bfdf5ac68523e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.752095 10917 tablet_service.cc:1467] Processing CreateTablet for tablet 8975b3e7ddc544bc888dd61fffc5313b (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1455921080 <= VALUES < 1492319107
I20260809 15:51:18.752182 11065 raft_consensus.cc:348] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.752899 11065 raft_consensus.cc:374] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:18.753275 10917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8975b3e7ddc544bc888dd61fffc5313b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.721312 10834 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 42) took 1412 ms. Trace:
I20260809 15:51:18.753679 10834 rpcz_store.cc:276] 0809 15:51:17.308943 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.489476 (+1180533us) service_pool.cc:227] Handling call
0809 15:51:18.511543 (+ 22067us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.511652 (+ 109us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.703707 (+192055us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.720848 (+ 17141us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.721256 (+ 408us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.749363 10836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8feeb57bf5cf4fd1bb17ab7f575222d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.755770 10834 tablet_service.cc:1467] Processing CreateTablet for tablet 369529a0023c4cf4a709bb1a480052b4 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260809 15:51:18.756886 10834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 369529a0023c4cf4a709bb1a480052b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.753203 11065 raft_consensus.cc:729] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:18.758200 11065 consensus_queue.cc:260] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.760399 11065 ts_tablet_manager.cc:1428] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260809 15:51:18.759214 11067 tablet_bootstrap.cc:492] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:18.762476 11067 ts_tablet_manager.cc:1397] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.264s user 0.037s sys 0.024s
I20260809 15:51:18.763695 10915 tablet_service.cc:1467] Processing CreateTablet for tablet 0b2b0df042e44725a7c91deae3df1956 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1601513188 <= VALUES < 1637911215
I20260809 15:51:18.764837 10915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b2b0df042e44725a7c91deae3df1956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.726060 10835 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 48) took 1309 ms. Trace:
I20260809 15:51:18.766793 10835 rpcz_store.cc:276] 0809 15:51:17.416767 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.663867 (+1247100us) service_pool.cc:227] Handling call
0809 15:51:18.682714 (+ 18847us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.682809 (+ 95us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.710424 (+ 27615us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.725495 (+ 15071us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.725996 (+ 501us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.706040 11003 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50526 (request call id 52) took 1093 ms. Trace:
I20260809 15:51:18.772923 11065 tablet_bootstrap.cc:492] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:18.775360 10827 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 46) took 1359 ms. Trace:
I20260809 15:51:18.775758 10827 rpcz_store.cc:276] 0809 15:51:17.415592 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.592512 (+1176920us) service_pool.cc:227] Handling call
0809 15:51:18.594268 (+ 1756us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.594363 (+ 95us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.694496 (+100133us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.774858 (+ 80362us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.775304 (+ 446us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.671440 11008 rpcz_store.cc:276] 0809 15:51:17.456822 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.197162 (+740340us) service_pool.cc:227] Handling call
0809 15:51:18.270397 (+ 73235us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.270520 (+ 123us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.590176 (+319656us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.625311 (+ 35135us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.625826 (+ 515us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.782939 11065 tablet_bootstrap.cc:654] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:18.787204 10835 tablet_service.cc:1467] Processing CreateTablet for tablet 34001ced69f34c6f930f5218eef46524 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260809 15:51:18.788542 10835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34001ced69f34c6f930f5218eef46524. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.769493 10919 tablet_service.cc:1467] Processing CreateTablet for tablet 93303d409ab940409d87013868417907 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1637911215 <= VALUES < 1674309242
I20260809 15:51:18.770287 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c8eaf50bcb3457bb4b1e30406cc349f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.800309 10998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f9b7d9e57934596b377ee8dc79ea23e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.802318 10919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93303d409ab940409d87013868417907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.771733 11003 rpcz_store.cc:276] 0809 15:51:17.612129 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.585623 (+973494us) service_pool.cc:227] Handling call
0809 15:51:18.597053 (+ 11430us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.597142 (+ 89us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.640507 (+ 43365us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.705553 (+ 65046us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.705984 (+ 431us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.798410 11067 raft_consensus.cc:348] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.813576 11067 raft_consensus.cc:374] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:18.814464 11067 raft_consensus.cc:729] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:18.816000 10827 tablet_service.cc:1467] Processing CreateTablet for tablet 279308d651cf45ff86edaae4483bcb79 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260809 15:51:18.817282 10827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 279308d651cf45ff86edaae4483bcb79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.815650 11067 consensus_queue.cc:260] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.829751 11073 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:18.830272 11073 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.830816 10833 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 41) took 1522 ms. Trace:
I20260809 15:51:18.831209 10833 rpcz_store.cc:276] 0809 15:51:17.308646 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.488225 (+1179579us) service_pool.cc:227] Handling call
0809 15:51:18.490808 (+ 2583us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.490897 (+ 89us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.723117 (+232220us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.830346 (+107229us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.830760 (+ 414us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.832234 11073 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:18.833161 10833 tablet_service.cc:1467] Processing CreateTablet for tablet bf6504ce1c0d4241bfb97eb553c8df35 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260809 15:51:18.834510 10833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6504ce1c0d4241bfb97eb553c8df35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.834607 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:18.835955 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:51:18.837545 10978 leader_election.cc:341] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
W20260809 15:51:18.838191 10978 leader_election.cc:341] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:18.838563 10978 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:18.839758 11073 raft_consensus.cc:2726] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:18.829427 10999 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50526 (request call id 53) took 1217 ms. Trace:
I20260809 15:51:18.840787 10999 rpcz_store.cc:276] 0809 15:51:17.612428 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.593683 (+981255us) service_pool.cc:227] Handling call
0809 15:51:18.595507 (+ 1824us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.595595 (+ 88us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.621548 (+ 25953us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.828937 (+207389us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.829369 (+ 432us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.840505 10909 tablet_service.cc:1467] Processing CreateTablet for tablet f4d1cc07da5b49ae95998fabc2323d24 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1674309242 <= VALUES < 1710707269
I20260809 15:51:18.848394 10912 tablet_service.cc:1467] Processing CreateTablet for tablet 0f891fe62df749108fa553cb0661a874 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1710707269 <= VALUES < 1747105296
I20260809 15:51:18.849694 10917 tablet_service.cc:1467] Processing CreateTablet for tablet 02f25d93293a40729fd751a19ab9adeb (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1747105296 <= VALUES < 1783503323
I20260809 15:51:18.851038 10912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f891fe62df749108fa553cb0661a874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.851747 10917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02f25d93293a40729fd751a19ab9adeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.831655 11072 tablet_bootstrap.cc:492] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:18.854575 11072 ts_tablet_manager.cc:1397] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.264s user 0.021s sys 0.029s
I20260809 15:51:18.854728 11083 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:18.855208 11083 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.855551 10828 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 44) took 1440 ms. Trace:
I20260809 15:51:18.855882 10828 rpcz_store.cc:276] 0809 15:51:17.414723 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.550644 (+1135921us) service_pool.cc:227] Handling call
0809 15:51:18.552763 (+ 2119us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.552874 (+ 111us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.713370 (+160496us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.855114 (+141744us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.855499 (+ 385us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.857161 11083 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:18.857915 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f4d1cc07da5b49ae95998fabc2323d24. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.859911 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:18.861107 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:51:18.862660 10891 leader_election.cc:341] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:18.857618 10828 tablet_service.cc:1467] Processing CreateTablet for tablet 241f07ed14b041cdb37e8ec792283d52 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260809 15:51:18.864079 10828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 241f07ed14b041cdb37e8ec792283d52. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 15:51:18.864399 10893 leader_election.cc:341] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:18.864792 10893 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:18.865901 11083 raft_consensus.cc:2726] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:18.857231 11072 raft_consensus.cc:348] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.869627 11072 raft_consensus.cc:374] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:18.869972 11072 raft_consensus.cc:729] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:18.870940 11072 consensus_queue.cc:260] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.873550 11072 ts_tablet_manager.cc:1428] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.019s user 0.007s sys 0.000s
I20260809 15:51:18.863739 10913 tablet_service.cc:1467] Processing CreateTablet for tablet 9d3f35b0ae8e4b02b72f832ca1518464 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1783503323 <= VALUES < 1819901350
I20260809 15:51:18.886652 10827 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 52) took 1468 ms. Trace:
I20260809 15:51:18.887022 10827 rpcz_store.cc:276] 0809 15:51:17.418576 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.815068 (+1396492us) service_pool.cc:227] Handling call
0809 15:51:18.816807 (+ 1739us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.816915 (+ 108us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.844580 (+ 27665us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.886085 (+ 41505us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.886566 (+ 481us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.888710 10833 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 53) took 1469 ms. Trace:
I20260809 15:51:18.888931 10830 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 49) took 1471 ms. Trace:
I20260809 15:51:18.889132 10833 rpcz_store.cc:276] 0809 15:51:17.418907 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.832242 (+1413335us) service_pool.cc:227] Handling call
0809 15:51:18.834091 (+ 1849us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.834182 (+ 91us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.841834 (+ 7652us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.888138 (+ 46304us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.888654 (+ 516us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.889278 10830 rpcz_store.cc:276] 0809 15:51:17.417386 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.671856 (+1254470us) service_pool.cc:227] Handling call
0809 15:51:18.691708 (+ 19852us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.691989 (+ 281us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.871647 (+179658us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.885821 (+ 14174us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.888862 (+ 3041us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2611}
I20260809 15:51:18.891359 10833 tablet_service.cc:1467] Processing CreateTablet for tablet 70c84efc43004b7b8ef2f9b869250509 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260809 15:51:18.891871 10830 tablet_service.cc:1467] Processing CreateTablet for tablet 6508f17228904310be3c95818973f3a1 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260809 15:51:18.892530 10833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70c84efc43004b7b8ef2f9b869250509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.893252 10830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6508f17228904310be3c95818973f3a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.895517 10919 tablet_service.cc:1467] Processing CreateTablet for tablet 8feeb57bf5cf4fd1bb17ab7f575222d0 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 2038289512 <= VALUES < 2074687539
I20260809 15:51:18.896963 10919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8feeb57bf5cf4fd1bb17ab7f575222d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.901396 11072 tablet_bootstrap.cc:492] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:18.901784 10913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d3f35b0ae8e4b02b72f832ca1518464. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.902336 10996 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50526 (request call id 54) took 1289 ms. Trace:
I20260809 15:51:18.902693 10996 rpcz_store.cc:276] 0809 15:51:17.612767 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.601208 (+988441us) service_pool.cc:227] Handling call
0809 15:51:18.634894 (+ 33686us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.642223 (+ 7329us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.759096 (+116873us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.853793 (+ 94697us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.902277 (+ 48484us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.906217 11065 tablet_bootstrap.cc:492] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:18.906708 11065 ts_tablet_manager.cc:1397] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.134s user 0.009s sys 0.005s
I20260809 15:51:18.907167 10827 tablet_service.cc:1467] Processing CreateTablet for tablet 1f9b7d9e57934596b377ee8dc79ea23e (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260809 15:51:18.907938 10910 tablet_service.cc:1467] Processing CreateTablet for tablet 369529a0023c4cf4a709bb1a480052b4 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1819901350 <= VALUES < 1856299377
I20260809 15:51:18.909162 10910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 369529a0023c4cf4a709bb1a480052b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.909312 11065 raft_consensus.cc:348] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:18.912470 10916 tablet_service.cc:1467] Processing CreateTablet for tablet 34001ced69f34c6f930f5218eef46524 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1856299377 <= VALUES < 1892697404
I20260809 15:51:18.913648 10916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34001ced69f34c6f930f5218eef46524. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.913813 11073 consensus_queue.cc:1035] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:18.908329 10827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f9b7d9e57934596b377ee8dc79ea23e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.926890 10829 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 47) took 1511 ms. Trace:
I20260809 15:51:18.910051 11065 raft_consensus.cc:374] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:18.927719 11065 raft_consensus.cc:729] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:18.927374 10829 rpcz_store.cc:276] 0809 15:51:17.415973 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.603948 (+1187975us) service_pool.cc:227] Handling call
0809 15:51:18.685562 (+ 81614us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.685702 (+ 140us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.838978 (+153276us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.901758 (+ 62780us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.926815 (+ 25057us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.933038 10917 tablet_service.cc:1467] Processing CreateTablet for tablet bf6504ce1c0d4241bfb97eb553c8df35 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1929095431 <= VALUES < 1965493458
I20260809 15:51:18.950363 10917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6504ce1c0d4241bfb97eb553c8df35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.893896 10834 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 50) took 1476 ms. Trace:
I20260809 15:51:18.952183 10834 rpcz_store.cc:276] 0809 15:51:17.417812 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.754904 (+1337092us) service_pool.cc:227] Handling call
0809 15:51:18.756502 (+ 1598us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.756582 (+ 80us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.888184 (+131602us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.893435 (+ 5251us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.893836 (+ 401us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.943943 10836 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 56) took 1523 ms. Trace:
I20260809 15:51:18.961716 11067 ts_tablet_manager.cc:1428] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.196s user 0.009s sys 0.000s
I20260809 15:51:18.961798 10909 tablet_service.cc:1467] Processing CreateTablet for tablet 279308d651cf45ff86edaae4483bcb79 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1892697404 <= VALUES < 1929095431
I20260809 15:51:18.945253 10918 tablet_service.cc:1467] Processing CreateTablet for tablet 241f07ed14b041cdb37e8ec792283d52 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 1965493458 <= VALUES < 2001891485
I20260809 15:51:18.968179 11072 tablet_bootstrap.cc:654] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:18.969437 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 241f07ed14b041cdb37e8ec792283d52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.971961 10835 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 51) took 1553 ms. Trace:
I20260809 15:51:18.972507 10835 rpcz_store.cc:276] 0809 15:51:17.418223 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.786171 (+1367948us) service_pool.cc:227] Handling call
0809 15:51:18.788026 (+ 1855us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.788142 (+ 116us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.960299 (+172157us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.971284 (+ 10985us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.971874 (+ 590us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.975402 10912 tablet_service.cc:1467] Processing CreateTablet for tablet 70c84efc43004b7b8ef2f9b869250509 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 2001891485 <= VALUES < 2038289512
I20260809 15:51:18.977227 10912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70c84efc43004b7b8ef2f9b869250509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:18.982510 11067 tablet_bootstrap.cc:492] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:18.987710 10832 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 43) took 1677 ms. Trace:
I20260809 15:51:18.962699 10836 rpcz_store.cc:276] 0809 15:51:17.420099 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.641693 (+1221594us) service_pool.cc:227] Handling call
0809 15:51:18.748922 (+107229us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.749029 (+ 107us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.880491 (+131462us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.923777 (+ 43286us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.943871 (+ 20094us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":19553}
I20260809 15:51:18.997258 10831 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 45) took 1582 ms. Trace:
I20260809 15:51:19.006157 10909 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 279308d651cf45ff86edaae4483bcb79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:19.006386 10831 rpcz_store.cc:276] 0809 15:51:17.415219 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.569257 (+1154038us) service_pool.cc:227] Handling call
0809 15:51:18.621602 (+ 52345us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.621699 (+ 97us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.709739 (+ 88040us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.985753 (+276014us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.997197 (+ 11444us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:19.006959 11087 consensus_queue.cc:1035] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 15:51:19.010600 10770 debug-util.cc:398] Leaking SignalData structure 0x7b080003cb40 after lost signal to thread 9556
W20260809 15:51:19.011663 10770 debug-util.cc:398] Leaking SignalData structure 0x7b0800033e40 after lost signal to thread 10773
I20260809 15:51:19.011499 11073 consensus_queue.cc:1035] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 15:51:19.012107 10770 debug-util.cc:398] Leaking SignalData structure 0x7b0800034020 after lost signal to thread 10797
I20260809 15:51:18.995039 10832 rpcz_store.cc:276] 0809 15:51:17.309243 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.495820 (+1186577us) service_pool.cc:227] Handling call
0809 15:51:18.497639 (+ 1819us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.497727 (+ 88us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.679432 (+181705us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:18.979195 (+299763us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:18.986703 (+ 7508us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:51:19.017100 10770 debug-util.cc:398] Leaking SignalData structure 0x7b08001c3820 after lost signal to thread 10959
I20260809 15:51:19.019876 11001 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50526 (request call id 56) took 1406 ms. Trace:
I20260809 15:51:19.027102 11001 rpcz_store.cc:276] 0809 15:51:17.613372 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.650612 (+1037240us) service_pool.cc:227] Handling call
0809 15:51:18.667242 (+ 16630us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.667336 (+ 94us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.833174 (+165838us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:19.016191 (+183017us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:19.019785 (+ 3594us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:19.029477 10910 tablet_service.cc:1467] Processing CreateTablet for tablet 6508f17228904310be3c95818973f3a1 (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 2074687539 <= VALUES < 2111085566
I20260809 15:51:19.026043 10833 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 55) took 1606 ms. Trace:
I20260809 15:51:19.035132 10833 rpcz_store.cc:276] 0809 15:51:17.419703 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.890300 (+1470597us) service_pool.cc:227] Handling call
0809 15:51:18.892121 (+ 1821us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.892209 (+ 88us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.905066 (+ 12857us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:19.025305 (+120239us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:19.025971 (+ 666us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:18.984473 11065 consensus_queue.cc:260] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.038069 11002 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50526 (request call id 50) took 1426 ms. Trace:
I20260809 15:51:19.039744 11002 rpcz_store.cc:276] 0809 15:51:17.611514 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.498837 (+887323us) service_pool.cc:227] Handling call
0809 15:51:18.560308 (+ 61471us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.560428 (+ 120us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.688903 (+128475us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:19.037142 (+348239us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:19.037989 (+ 847us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:19.040652 11065 ts_tablet_manager.cc:1428] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.134s user 0.008s sys 0.001s
I20260809 15:51:19.041895 11065 tablet_bootstrap.cc:492] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:19.028784 11000 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50526 (request call id 55) took 1415 ms. Trace:
I20260809 15:51:19.042268 10913 tablet_service.cc:1467] Processing CreateTablet for tablet 1f9b7d9e57934596b377ee8dc79ea23e (DEFAULT_TABLE table=test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]), partition=RANGE (key) PARTITION 2111085566 <= VALUES
I20260809 15:51:19.021283 11006 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50526 (request call id 57) took 1407 ms. Trace:
I20260809 15:51:19.042668 11000 rpcz_store.cc:276] 0809 15:51:17.613086 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.605670 (+992584us) service_pool.cc:227] Handling call
0809 15:51:18.662140 (+ 56470us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.662258 (+ 118us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.895805 (+233547us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:19.027968 (+132163us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:19.028689 (+ 721us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:19.054689 10910 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6508f17228904310be3c95818973f3a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:19.061796 10830 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 57) took 1641 ms. Trace:
I20260809 15:51:19.064597 10830 rpcz_store.cc:276] 0809 15:51:17.420463 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.891029 (+1470566us) service_pool.cc:227] Handling call
0809 15:51:18.892600 (+ 1571us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.892682 (+ 82us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.979590 (+ 86908us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:19.061192 (+ 81602us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:19.061725 (+ 533us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:19.068264 11070 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:19.043846 11006 rpcz_store.cc:276] 0809 15:51:17.613700 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.666459 (+1052759us) service_pool.cc:227] Handling call
0809 15:51:18.668219 (+ 1760us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.668303 (+ 84us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.813122 (+144819us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:19.020669 (+207547us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:19.021214 (+ 545us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:19.068907 11070 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.075047 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:19.077522 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:19.078461 10930 raft_consensus.cc:2454] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:19.081065 10811 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
W20260809 15:51:19.083004 10809 leader_election.cc:341] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:19.093783 11067 tablet_bootstrap.cc:654] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:19.093688 10916 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54216 (request call id 51) took 1002 ms. Trace:
I20260809 15:51:19.102471 10916 rpcz_store.cc:276] 0809 15:51:18.091297 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.911647 (+820350us) service_pool.cc:227] Handling call
0809 15:51:18.913220 (+ 1573us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.913313 (+ 93us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.925129 (+ 11816us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:19.093016 (+167887us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:19.093617 (+ 601us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:19.085508 11070 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:19.086516 10998 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:50526 (request call id 58) took 1471 ms. Trace:
I20260809 15:51:19.097790 10913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f9b7d9e57934596b377ee8dc79ea23e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:19.114500 10914 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54216 (request call id 43) took 1033 ms. Trace:
I20260809 15:51:19.115602 11070 raft_consensus.cc:2781] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:19.115087 10914 rpcz_store.cc:276] 0809 15:51:18.081559 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.746956 (+665397us) service_pool.cc:227] Handling call
0809 15:51:18.750583 (+ 3627us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.750701 (+ 118us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:19.076054 (+325353us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:19.113791 (+ 37737us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:19.114420 (+ 629us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:19.129585 11070 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:19.108150 10998 rpcz_store.cc:276] 0809 15:51:17.613989 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.740644 (+1126655us) service_pool.cc:227] Handling call
0809 15:51:18.771919 (+ 31275us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.799948 (+ 28029us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.967272 (+167324us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:19.043771 (+ 76499us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:19.085648 (+ 41877us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:19.153280 11065 tablet_bootstrap.cc:654] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
W20260809 15:51:19.131502 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 51284.7us
I20260809 15:51:19.146597 11070 raft_consensus.cc:3037] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:19.173056 11109 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:19.180783 10827 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 58) took 1760 ms. Trace:
I20260809 15:51:19.181137 11072 tablet_bootstrap.cc:492] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:19.181703 11072 ts_tablet_manager.cc:1397] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.280s user 0.023s sys 0.011s
I20260809 15:51:19.182211 10915 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54216 (request call id 44) took 1100 ms. Trace:
I20260809 15:51:19.182690 10915 rpcz_store.cc:276] 0809 15:51:18.081924 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.762880 (+680956us) service_pool.cc:227] Handling call
0809 15:51:18.764432 (+ 1552us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.764515 (+ 83us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:18.917660 (+153145us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:19.181316 (+263656us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:19.182041 (+ 725us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:19.186062 11072 raft_consensus.cc:348] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } }
I20260809 15:51:19.187036 11072 raft_consensus.cc:374] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.187494 11072 raft_consensus.cc:729] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.188730 11072 consensus_queue.cc:260] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } }
I20260809 15:51:19.181192 10827 rpcz_store.cc:276] 0809 15:51:17.420818 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.906296 (+1485478us) service_pool.cc:227] Handling call
0809 15:51:18.907890 (+ 1594us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.907993 (+ 103us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:19.066295 (+158302us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:19.180100 (+113805us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:19.180727 (+ 627us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:19.190271 10919 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54216 (request call id 56) took 1094 ms. Trace:
I20260809 15:51:19.190841 10919 rpcz_store.cc:276] 0809 15:51:18.095785 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.894640 (+798855us) service_pool.cc:227] Handling call
0809 15:51:18.896406 (+ 1766us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.896611 (+ 205us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:19.092484 (+195873us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:19.189403 (+ 96919us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:19.190183 (+ 780us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:19.187140 11109 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.192471 10828 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:45572 (request call id 54) took 1773 ms. Trace:
I20260809 15:51:19.195925 10917 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54216 (request call id 53) took 1104 ms. Trace:
I20260809 15:51:19.200912 10828 rpcz_store.cc:276] 0809 15:51:17.419289 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.856756 (+1437467us) service_pool.cc:227] Handling call
0809 15:51:18.863642 (+ 6886us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.863734 (+ 92us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:19.026003 (+162269us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:19.191593 (+165590us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:19.192327 (+ 734us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:19.204331 10912 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54216 (request call id 55) took 1111 ms. Trace:
W20260809 15:51:19.208000 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 83055.6us
I20260809 15:51:19.216642 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:51:19.220243 10811 leader_election.cc:341] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:19.220889 11070 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.224555 11109 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:19.225673 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:19.226879 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
W20260809 15:51:19.227044 10809 leader_election.cc:341] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:19.227602 10931 raft_consensus.cc:3037] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:19.227545 10809 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:19.228817 11109 raft_consensus.cc:2726] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:19.231599 10909 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54216 (request call id 52) took 1140 ms. Trace:
I20260809 15:51:19.232583 11072 ts_tablet_manager.cc:1428] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.051s user 0.011s sys 0.001s
I20260809 15:51:19.233736 11072 tablet_bootstrap.cc:492] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:19.232014 10909 rpcz_store.cc:276] 0809 15:51:18.091647 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.927553 (+835906us) service_pool.cc:227] Handling call
0809 15:51:18.987430 (+ 59877us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.990123 (+ 2693us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:19.120896 (+130773us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:19.230977 (+110081us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:19.231535 (+ 558us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:19.202262 10917 rpcz_store.cc:276] 0809 15:51:18.091973 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.931645 (+839672us) service_pool.cc:227] Handling call
0809 15:51:18.949572 (+ 17927us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.949693 (+ 121us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:19.081788 (+132095us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:19.195266 (+113478us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:19.195868 (+ 602us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:19.235821 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:19.209280 10912 rpcz_store.cc:276] 0809 15:51:18.092599 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:18.973694 (+881095us) service_pool.cc:227] Handling call
0809 15:51:18.976485 (+ 2791us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:18.976653 (+ 168us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:19.181299 (+204646us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:19.199267 (+ 17968us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:19.204277 (+ 5010us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:51:19.237555 10809 leader_election.cc:341] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:19.238600 11070 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:19.227083 11067 tablet_bootstrap.cc:492] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:19.241462 11067 ts_tablet_manager.cc:1397] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.259s user 0.012s sys 0.013s
I20260809 15:51:19.243201 11065 tablet_bootstrap.cc:492] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:19.242367 10931 raft_consensus.cc:2454] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:19.243815 11065 ts_tablet_manager.cc:1397] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.202s user 0.013s sys 0.006s
I20260809 15:51:19.245155 10811 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:19.245740 11067 raft_consensus.cc:348] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.246996 11109 raft_consensus.cc:2781] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:19.247459 11067 raft_consensus.cc:374] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.247634 11109 raft_consensus.cc:686] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:19.248226 11067 raft_consensus.cc:729] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.249487 11109 consensus_queue.cc:237] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.249891 11067 consensus_queue.cc:260] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:19.254820 10978 consensus_peers.cc:487] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa -> Peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Couldn't send request to peer 20730d75ad954a82b126148f8152a9f3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:19.261200 10913 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.CreateTablet from 127.0.0.1:54216 (request call id 58) took 1164 ms. Trace:
I20260809 15:51:19.261734 10913 rpcz_store.cc:276] 0809 15:51:18.096586 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:19.040929 (+944343us) service_pool.cc:227] Handling call
0809 15:51:19.043244 (+ 2315us) ts_tablet_manager.cc:663] Acquired tablet manager lock
0809 15:51:19.043361 (+ 117us) ts_tablet_manager.cc:676] Creating new metadata...
0809 15:51:19.202189 (+158828us) tablet_metadata.cc:641] Metadata flushed
0809 15:51:19.239288 (+ 37099us) tablet_replica.cc:205] Creating consensus instance
0809 15:51:19.261130 (+ 21842us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:51:19.265498 11067 ts_tablet_manager.cc:1428] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.024s user 0.009s sys 0.002s
I20260809 15:51:19.266574 11067 tablet_bootstrap.cc:492] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:19.264103 11065 raft_consensus.cc:348] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.265131 11072 tablet_bootstrap.cc:654] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:19.273073 11065 raft_consensus.cc:374] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.273519 11065 raft_consensus.cc:729] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.277523 11065 consensus_queue.cc:260] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.287353 11067 tablet_bootstrap.cc:654] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:19.292198 11072 tablet_bootstrap.cc:492] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:19.292698 11072 ts_tablet_manager.cc:1397] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.059s user 0.008s sys 0.010s
I20260809 15:51:19.297497 11072 raft_consensus.cc:348] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.298370 11072 raft_consensus.cc:374] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.298730 11072 raft_consensus.cc:729] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.302294 11072 consensus_queue.cc:260] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.303085 10735 catalog_manager.cc:5495] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 0 to 1, leader changed from <none> to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:19.308991 11072 ts_tablet_manager.cc:1428] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.016s user 0.007s sys 0.003s
I20260809 15:51:19.310066 11072 tablet_bootstrap.cc:492] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:19.324853 11065 ts_tablet_manager.cc:1428] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.080s user 0.009s sys 0.000s
I20260809 15:51:19.325031 11072 tablet_bootstrap.cc:654] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:19.351619 11083 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:19.352284 11083 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.353197 11072 tablet_bootstrap.cc:492] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:19.353657 11072 ts_tablet_manager.cc:1397] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.044s user 0.008s sys 0.010s
I20260809 15:51:19.356715 11072 raft_consensus.cc:348] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.356948 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:51:19.358976 10891 leader_election.cc:341] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:19.359067 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:19.359290 11072 raft_consensus.cc:374] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.360533 11072 raft_consensus.cc:729] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.360956 11083 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:51:19.361032 10893 leader_election.cc:341] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:19.361971 10893 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:19.362830 11072 consensus_queue.cc:260] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.363649 11083 raft_consensus.cc:2726] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:19.369608 11065 tablet_bootstrap.cc:492] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:19.379757 11072 ts_tablet_manager.cc:1428] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.026s user 0.011s sys 0.004s
I20260809 15:51:19.380589 11067 tablet_bootstrap.cc:492] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:19.381414 11067 ts_tablet_manager.cc:1397] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.115s user 0.025s sys 0.022s
I20260809 15:51:19.383628 11065 tablet_bootstrap.cc:654] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:19.385911 11072 tablet_bootstrap.cc:492] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:19.385871 11067 raft_consensus.cc:348] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.387176 11067 raft_consensus.cc:374] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.387602 11067 raft_consensus.cc:729] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.388880 11067 consensus_queue.cc:260] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.393693 11067 ts_tablet_manager.cc:1428] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.012s user 0.002s sys 0.006s
I20260809 15:51:19.395417 11067 tablet_bootstrap.cc:492] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:19.397702 11072 tablet_bootstrap.cc:654] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:19.406298 11126 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:19.406970 11126 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.407713 11067 tablet_bootstrap.cc:654] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:19.410997 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:19.412678 10893 leader_election.cc:341] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:19.419716 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:19.420440 11018 raft_consensus.cc:2454] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:19.421958 11072 tablet_bootstrap.cc:492] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:19.422361 11126 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:19.422561 11072 ts_tablet_manager.cc:1397] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.037s user 0.013s sys 0.023s
I20260809 15:51:19.421766 10891 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:19.423787 11126 raft_consensus.cc:2781] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:19.424340 11126 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:19.424868 11126 raft_consensus.cc:3037] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:19.425611 11067 tablet_bootstrap.cc:492] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:19.426331 11067 ts_tablet_manager.cc:1397] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.031s user 0.008s sys 0.019s
I20260809 15:51:19.426393 11072 raft_consensus.cc:348] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.427663 11072 raft_consensus.cc:374] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.428205 11072 raft_consensus.cc:729] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.429215 11072 consensus_queue.cc:260] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.431123 11067 raft_consensus.cc:348] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.432134 11067 raft_consensus.cc:374] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.432664 11067 raft_consensus.cc:729] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.433636 11067 consensus_queue.cc:260] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.435210 11065 tablet_bootstrap.cc:492] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:19.435813 11065 ts_tablet_manager.cc:1397] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.099s user 0.003s sys 0.022s
I20260809 15:51:19.437398 11067 ts_tablet_manager.cc:1428] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.011s user 0.008s sys 0.000s
I20260809 15:51:19.438881 11067 tablet_bootstrap.cc:492] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:19.443547 11072 ts_tablet_manager.cc:1428] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.021s user 0.008s sys 0.001s
I20260809 15:51:19.444934 11126 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.448571 11067 tablet_bootstrap.cc:654] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:19.449033 11083 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:19.449183 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:19.449718 11083 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } }
I20260809 15:51:19.452316 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:19.453068 11083 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:19.455351 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:51:19.457240 10891 leader_election.cc:341] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:19.453199 11018 raft_consensus.cc:3037] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:19.459957 11072 tablet_bootstrap.cc:492] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
W20260809 15:51:19.461117 10893 leader_election.cc:341] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:19.462517 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:19.464224 10893 leader_election.cc:341] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:19.464920 10893 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:19.466197 11083 raft_consensus.cc:2726] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:19.443076 11065 raft_consensus.cc:348] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.467737 11065 raft_consensus.cc:374] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.469156 11065 raft_consensus.cc:729] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.470268 11065 consensus_queue.cc:260] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.471482 11072 tablet_bootstrap.cc:654] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:19.467829 11126 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:19.477821 11067 tablet_bootstrap.cc:492] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:19.480310 11067 ts_tablet_manager.cc:1397] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.042s user 0.010s sys 0.020s
I20260809 15:51:19.483410 11065 ts_tablet_manager.cc:1428] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.047s user 0.001s sys 0.008s
I20260809 15:51:19.485419 11067 raft_consensus.cc:348] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.486478 11067 raft_consensus.cc:374] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.486934 11067 raft_consensus.cc:729] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.488138 11067 consensus_queue.cc:260] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.491652 11067 ts_tablet_manager.cc:1428] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.011s user 0.009s sys 0.001s
I20260809 15:51:19.479310 11018 raft_consensus.cc:2454] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:19.493386 10891 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:19.495267 11126 raft_consensus.cc:2781] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:19.496170 11126 raft_consensus.cc:686] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:19.496655 11067 tablet_bootstrap.cc:492] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:19.497354 11126 consensus_queue.cc:237] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.500811 11065 tablet_bootstrap.cc:492] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:19.515676 11072 tablet_bootstrap.cc:492] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:19.516263 11072 ts_tablet_manager.cc:1397] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.057s user 0.018s sys 0.017s
I20260809 15:51:19.520411 11072 raft_consensus.cc:348] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.521477 11072 raft_consensus.cc:374] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.521842 11072 raft_consensus.cc:729] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.522302 11065 tablet_bootstrap.cc:654] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:19.522939 11072 consensus_queue.cc:260] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.526211 11072 ts_tablet_manager.cc:1428] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260809 15:51:19.527104 11072 tablet_bootstrap.cc:492] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:19.533242 11067 tablet_bootstrap.cc:654] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:19.533675 10930 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:51:19.539377 10789 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 15:51:19.550529 10737 catalog_manager.cc:5495] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:19.555650 11065 tablet_bootstrap.cc:492] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:19.556181 11065 ts_tablet_manager.cc:1397] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.056s user 0.007s sys 0.021s
I20260809 15:51:19.551702 11072 tablet_bootstrap.cc:654] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:19.560714 11065 raft_consensus.cc:348] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.561793 11065 raft_consensus.cc:374] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.562323 11065 raft_consensus.cc:729] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.563544 11065 consensus_queue.cc:260] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.571446 11067 tablet_bootstrap.cc:492] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:19.571969 11067 ts_tablet_manager.cc:1397] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.076s user 0.011s sys 0.012s
I20260809 15:51:19.575315 11067 raft_consensus.cc:348] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.576116 11067 raft_consensus.cc:374] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.576313 11065 ts_tablet_manager.cc:1428] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.020s user 0.006s sys 0.005s
I20260809 15:51:19.582437 11065 tablet_bootstrap.cc:492] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:19.576432 11067 raft_consensus.cc:729] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.586140 11067 consensus_queue.cc:260] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.588833 11067 ts_tablet_manager.cc:1428] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20260809 15:51:19.589879 11067 tablet_bootstrap.cc:492] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:19.593305 11065 tablet_bootstrap.cc:654] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:19.603737 11072 tablet_bootstrap.cc:492] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:19.604122 11067 tablet_bootstrap.cc:654] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:19.604244 11072 ts_tablet_manager.cc:1397] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.077s user 0.010s sys 0.007s
I20260809 15:51:19.606701 11072 raft_consensus.cc:348] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.607211 11072 raft_consensus.cc:374] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.607421 11072 raft_consensus.cc:729] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.607985 11072 consensus_queue.cc:260] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.611475 11067 tablet_bootstrap.cc:492] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:19.611917 11067 ts_tablet_manager.cc:1397] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.022s user 0.010s sys 0.006s
I20260809 15:51:19.614504 11065 tablet_bootstrap.cc:492] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:19.615060 11065 ts_tablet_manager.cc:1397] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.033s user 0.015s sys 0.003s
I20260809 15:51:19.614696 11067 raft_consensus.cc:348] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.615667 11067 raft_consensus.cc:374] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.615958 11067 raft_consensus.cc:729] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.616608 11067 consensus_queue.cc:260] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.618479 11065 raft_consensus.cc:348] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.619119 11067 ts_tablet_manager.cc:1428] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.007s user 0.006s sys 0.002s
I20260809 15:51:19.619283 11065 raft_consensus.cc:374] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.619791 11065 raft_consensus.cc:729] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.620553 11065 consensus_queue.cc:260] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.621109 11072 ts_tablet_manager.cc:1428] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.016s user 0.005s sys 0.001s
I20260809 15:51:19.622519 11065 ts_tablet_manager.cc:1428] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 15:51:19.623327 11065 tablet_bootstrap.cc:492] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:19.631390 11065 tablet_bootstrap.cc:654] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:19.621933 11072 tablet_bootstrap.cc:492] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:19.634269 11109 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:19.635314 11109 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.637746 11067 tablet_bootstrap.cc:492] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:19.638477 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:19.639961 11072 tablet_bootstrap.cc:654] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:19.645540 11067 tablet_bootstrap.cc:654] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
W20260809 15:51:19.647231 10811 leader_election.cc:341] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:19.648221 11109 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:19.655654 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:19.656523 11018 raft_consensus.cc:2454] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:19.658282 11126 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:19.659070 11126 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.660451 10809 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:19.662237 11109 raft_consensus.cc:2781] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:19.662571 11109 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:19.662982 11109 raft_consensus.cc:3037] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:19.663357 11126 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:19.663129 11065 tablet_bootstrap.cc:492] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:19.665055 11065 ts_tablet_manager.cc:1397] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.042s user 0.017s sys 0.011s
I20260809 15:51:19.667383 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:19.667428 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:19.667714 11065 raft_consensus.cc:348] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.668227 11018 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:19.668828 11065 raft_consensus.cc:374] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.669170 11065 raft_consensus.cc:729] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
W20260809 15:51:19.669296 10893 leader_election.cc:341] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:19.670048 10891 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:19.669958 11065 consensus_queue.cc:260] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.671579 11126 raft_consensus.cc:2781] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:19.672228 11126 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:19.672610 11065 ts_tablet_manager.cc:1428] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 15:51:19.672843 11126 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:19.674201 11065 tablet_bootstrap.cc:492] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:19.682003 11065 tablet_bootstrap.cc:654] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:19.685899 11070 consensus_queue.cc:1035] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:19.689088 11115 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:19.689663 11115 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.691797 11109 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.694036 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:19.694904 10848 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:19.699250 11072 tablet_bootstrap.cc:492] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:19.699831 11072 ts_tablet_manager.cc:1397] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.078s user 0.010s sys 0.012s
I20260809 15:51:19.703227 11072 raft_consensus.cc:348] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.704378 11072 raft_consensus.cc:374] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.704703 11072 raft_consensus.cc:729] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.705178 11072 consensus_queue.cc:260] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.697214 10978 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:19.707034 11070 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:19.707490 11070 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.708124 11073 raft_consensus.cc:2781] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:19.708616 11073 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:19.709044 11073 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:19.711320 11070 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:19.713023 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:19.713757 11018 raft_consensus.cc:2454] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:19.715443 10809 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:19.716689 11113 raft_consensus.cc:2781] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:19.717208 11113 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:19.716915 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:19.718472 10931 raft_consensus.cc:2454] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:19.717720 11113 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:19.719846 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:19.721160 10929 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:19.722755 11126 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.726477 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:19.727309 11018 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:19.728672 11115 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:19.730677 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:51:19.732496 10893 leader_election.cc:341] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:19.736325 11072 ts_tablet_manager.cc:1428] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.036s user 0.007s sys 0.000s
I20260809 15:51:19.737892 11072 tablet_bootstrap.cc:492] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:19.729642 11113 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.742443 11113 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:19.743153 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:19.743615 11017 raft_consensus.cc:3037] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:19.743824 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:19.743865 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:19.744472 10929 raft_consensus.cc:3037] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:19.744573 10931 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:19.746635 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:19.747157 11016 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:19.745271 11109 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:19.739310 11126 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:19.749951 11065 tablet_bootstrap.cc:492] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:19.750530 11065 ts_tablet_manager.cc:1397] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.077s user 0.015s sys 0.033s
I20260809 15:51:19.755533 11115 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:19.755677 11065 raft_consensus.cc:348] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.756616 11065 raft_consensus.cc:374] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.756206 11115 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.747339 11018 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:19.759030 11115 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:19.758893 10891 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:19.759859 11065 raft_consensus.cc:729] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.761214 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:19.761113 11065 consensus_queue.cc:260] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.762230 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:19.763190 10978 leader_election.cc:341] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:19.763628 10848 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:19.762907 11113 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:19.764839 11067 tablet_bootstrap.cc:492] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:19.765376 11067 ts_tablet_manager.cc:1397] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.128s user 0.013s sys 0.006s
I20260809 15:51:19.764999 11113 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.765017 10978 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:19.770470 11115 raft_consensus.cc:2781] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:19.770954 11115 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:19.771482 11115 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:19.771656 11067 raft_consensus.cc:348] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.771968 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:19.772490 11067 raft_consensus.cc:374] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.772795 11067 raft_consensus.cc:729] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.772764 10930 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:19.773814 11067 consensus_queue.cc:260] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.774724 10811 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:19.781556 11115 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.784312 11067 ts_tablet_manager.cc:1428] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.019s user 0.006s sys 0.003s
I20260809 15:51:19.785485 11109 raft_consensus.cc:2781] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:19.786650 11067 tablet_bootstrap.cc:492] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:19.767061 11126 raft_consensus.cc:2781] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:19.785547 10931 raft_consensus.cc:2454] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:19.786170 11109 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:19.792539 11109 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:19.798950 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:19.799829 10848 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:19.799736 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:51:19.801954 10809 leader_election.cc:341] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:19.790751 11126 raft_consensus.cc:686] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:19.803560 11113 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:19.804167 11065 ts_tablet_manager.cc:1428] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.053s user 0.014s sys 0.006s
I20260809 15:51:19.804077 11126 consensus_queue.cc:237] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.806671 11065 tablet_bootstrap.cc:492] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:19.814555 11073 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.816757 11073 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:19.808341 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:19.818162 11115 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:19.810932 11016 raft_consensus.cc:2454] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:19.812232 11017 raft_consensus.cc:2454] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:19.809531 10929 raft_consensus.cc:2454] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:19.821720 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:19.822512 10847 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:19.822634 10809 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:19.823768 11113 raft_consensus.cc:2781] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:19.824364 11113 raft_consensus.cc:686] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:19.825515 11113 consensus_queue.cc:237] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:19.821080 10978 leader_election.cc:341] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:19.812294 11067 tablet_bootstrap.cc:654] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:19.825522 11065 tablet_bootstrap.cc:654] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:19.826599 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:19.828241 10930 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:19.829015 11017 raft_consensus.cc:3037] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:19.830288 10809 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:19.833482 10848 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:19.836048 11109 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.840641 10978 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
W20260809 15:51:19.829519 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 54877.7us
I20260809 15:51:19.840233 11070 raft_consensus.cc:2781] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:19.841542 10847 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:19.841869 11070 raft_consensus.cc:686] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:19.842418 11115 raft_consensus.cc:2781] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:19.847795 11115 raft_consensus.cc:686] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:19.851413 10978 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:19.851626 11115 consensus_queue.cc:237] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.854768 11073 raft_consensus.cc:2781] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:19.855381 11073 raft_consensus.cc:686] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:19.856719 11073 consensus_queue.cc:237] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.859212 11109 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:19.859905 11149 consensus_queue.cc:1035] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260809 15:51:19.861871 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:19.866935 11072 tablet_bootstrap.cc:654] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
W20260809 15:51:19.868989 10809 leader_election.cc:341] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:19.857717 11070 consensus_queue.cc:237] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.873836 10930 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:19.875288 11067 tablet_bootstrap.cc:492] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:19.876104 11067 ts_tablet_manager.cc:1397] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.090s user 0.008s sys 0.006s
I20260809 15:51:19.879515 11067 raft_consensus.cc:348] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.879994 11067 raft_consensus.cc:374] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.880285 11067 raft_consensus.cc:729] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.882690 11067 consensus_queue.cc:260] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.892295 11067 ts_tablet_manager.cc:1428] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.016s user 0.005s sys 0.003s
I20260809 15:51:19.903280 11067 tablet_bootstrap.cc:492] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:19.907878 10736 catalog_manager.cc:5495] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:19.925513 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:19.926569 10930 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:19.933030 10737 catalog_manager.cc:5495] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 0 to 1, leader changed from <none> to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:19.934556 10737 catalog_manager.cc:5495] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 0 to 1, leader changed from <none> to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:19.940362 10735 catalog_manager.cc:5495] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:19.949553 11126 consensus_queue.cc:1035] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:19.966972 11065 tablet_bootstrap.cc:492] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:19.967483 11065 ts_tablet_manager.cc:1397] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.161s user 0.012s sys 0.006s
I20260809 15:51:19.971915 10930 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:19.973093 10811 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:19.973390 11067 tablet_bootstrap.cc:654] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:19.973664 11065 raft_consensus.cc:348] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.974404 11070 raft_consensus.cc:2781] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:19.975181 11070 raft_consensus.cc:686] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:19.974604 11065 raft_consensus.cc:374] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.976878 11065 raft_consensus.cc:729] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.976519 11070 consensus_queue.cc:237] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.977653 11065 consensus_queue.cc:260] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.978472 11072 tablet_bootstrap.cc:492] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:19.978969 11072 ts_tablet_manager.cc:1397] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.241s user 0.010s sys 0.045s
I20260809 15:51:19.983783 11072 raft_consensus.cc:348] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.984436 11072 raft_consensus.cc:374] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:19.984844 11072 raft_consensus.cc:729] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:19.985548 11072 consensus_queue.cc:260] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:19.989352 11072 ts_tablet_manager.cc:1428] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.010s user 0.005s sys 0.003s
I20260809 15:51:19.990584 11072 tablet_bootstrap.cc:492] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:19.990967 11065 ts_tablet_manager.cc:1428] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.023s user 0.011s sys 0.000s
I20260809 15:51:19.992475 11065 tablet_bootstrap.cc:492] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:20.002229 10737 catalog_manager.cc:5495] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
W20260809 15:51:20.007341 10893 consensus_peers.cc:487] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 -> Peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Couldn't send request to peer a53441872c0c416b8e2f26f52399f536. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:20.028353 11072 tablet_bootstrap.cc:654] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:20.029603 11065 tablet_bootstrap.cc:654] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:20.029089 10734 catalog_manager.cc:5495] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 0 to 1, leader changed from <none> to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:20.034433 11067 tablet_bootstrap.cc:492] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:20.034924 11067 ts_tablet_manager.cc:1397] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.135s user 0.013s sys 0.027s
I20260809 15:51:20.046844 11067 raft_consensus.cc:348] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.047556 11067 raft_consensus.cc:374] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:20.047806 11067 raft_consensus.cc:729] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:20.048998 11065 tablet_bootstrap.cc:492] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:20.049466 11065 ts_tablet_manager.cc:1397] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.057s user 0.009s sys 0.006s
I20260809 15:51:20.052191 11067 consensus_queue.cc:260] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.053090 11065 raft_consensus.cc:348] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.053771 11065 raft_consensus.cc:374] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:20.054034 11065 raft_consensus.cc:729] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:20.054647 11065 consensus_queue.cc:260] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.057010 11065 ts_tablet_manager.cc:1428] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260809 15:51:20.058152 11065 tablet_bootstrap.cc:492] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:20.063514 11067 ts_tablet_manager.cc:1428] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.028s user 0.006s sys 0.004s
I20260809 15:51:20.064410 11067 tablet_bootstrap.cc:492] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:20.065236 11065 tablet_bootstrap.cc:654] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:20.072021 11067 tablet_bootstrap.cc:654] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:20.073505 11072 tablet_bootstrap.cc:492] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:20.074196 11072 ts_tablet_manager.cc:1397] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.084s user 0.017s sys 0.003s
I20260809 15:51:20.078011 11072 raft_consensus.cc:348] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.078941 11072 raft_consensus.cc:374] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:20.079289 11072 raft_consensus.cc:729] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:20.080173 11072 consensus_queue.cc:260] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.084022 11072 ts_tablet_manager.cc:1428] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.009s user 0.007s sys 0.001s
I20260809 15:51:20.085232 11065 tablet_bootstrap.cc:492] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:20.085721 11065 ts_tablet_manager.cc:1397] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.028s user 0.011s sys 0.006s
I20260809 15:51:20.086061 11072 tablet_bootstrap.cc:492] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:20.088243 11067 tablet_bootstrap.cc:492] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:20.089041 11067 ts_tablet_manager.cc:1397] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.025s user 0.017s sys 0.005s
I20260809 15:51:20.088688 11065 raft_consensus.cc:348] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } }
I20260809 15:51:20.089738 11065 raft_consensus.cc:374] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:20.090014 11065 raft_consensus.cc:729] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:20.091143 11065 consensus_queue.cc:260] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } }
I20260809 15:51:20.092940 11067 raft_consensus.cc:348] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.093740 11067 raft_consensus.cc:374] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:20.094069 11067 raft_consensus.cc:729] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:20.094934 11067 consensus_queue.cc:260] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.096446 11072 tablet_bootstrap.cc:654] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:20.097918 11067 ts_tablet_manager.cc:1428] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.008s user 0.000s sys 0.006s
I20260809 15:51:20.098160 11106 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:20.099282 11067 tablet_bootstrap.cc:492] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:20.099799 11065 ts_tablet_manager.cc:1428] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.014s user 0.008s sys 0.000s
I20260809 15:51:20.099308 11106 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.100844 11065 tablet_bootstrap.cc:492] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:20.102072 11106 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:20.104672 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:20.105628 10847 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:20.108768 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:20.109253 11067 tablet_bootstrap.cc:654] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:20.109397 10930 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:20.111392 10978 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:20.112335 11106 raft_consensus.cc:2781] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:20.112720 11106 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:20.113410 11106 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:20.116586 11065 tablet_bootstrap.cc:654] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:20.132273 11070 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:20.132948 11070 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.138100 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:20.141917 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:20.143782 10809 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:20.144903 11113 raft_consensus.cc:2726] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:20.145872 11070 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:20.146150 11073 raft_consensus.cc:482] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:20.146775 11073 raft_consensus.cc:504] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.149214 11073 leader_election.cc:288] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:20.149502 11067 tablet_bootstrap.cc:492] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:20.149943 11067 ts_tablet_manager.cc:1397] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.051s user 0.005s sys 0.024s
I20260809 15:51:20.151172 11106 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.151911 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:51:20.154367 10978 leader_election.cc:341] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:20.155090 11067 raft_consensus.cc:348] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } }
I20260809 15:51:20.155413 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:20.155750 11067 raft_consensus.cc:374] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:20.155664 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:20.156108 11067 raft_consensus.cc:729] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:20.156641 10848 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:51:20.157078 10978 leader_election.cc:341] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:20.156945 11067 consensus_queue.cc:260] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } }
I20260809 15:51:20.160110 11067 ts_tablet_manager.cc:1428] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.010s user 0.007s sys 0.001s
I20260809 15:51:20.161345 11067 tablet_bootstrap.cc:492] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:20.157716 10978 leader_election.cc:302] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:20.163807 11073 raft_consensus.cc:2726] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:20.170730 11067 tablet_bootstrap.cc:654] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:20.176666 10848 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:20.178365 10978 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:20.179212 11073 raft_consensus.cc:2781] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:20.179836 11072 tablet_bootstrap.cc:492] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:20.179730 11106 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:20.179843 11073 raft_consensus.cc:686] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:20.180474 11072 ts_tablet_manager.cc:1397] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.095s user 0.019s sys 0.011s
I20260809 15:51:20.181082 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:20.181988 10930 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:20.181656 11073 consensus_queue.cc:237] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.184341 11072 raft_consensus.cc:348] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.185158 11072 raft_consensus.cc:374] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:20.185477 11072 raft_consensus.cc:729] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:20.186374 11072 consensus_queue.cc:260] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.189345 11072 ts_tablet_manager.cc:1428] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.009s user 0.007s sys 0.002s
I20260809 15:51:20.190356 11072 tablet_bootstrap.cc:492] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:20.191881 11065 tablet_bootstrap.cc:492] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:20.192471 11065 ts_tablet_manager.cc:1397] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.092s user 0.021s sys 0.025s
I20260809 15:51:20.196637 11065 raft_consensus.cc:348] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.204314 11065 raft_consensus.cc:374] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:20.204725 11065 raft_consensus.cc:729] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:20.205693 11065 consensus_queue.cc:260] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.203572 11072 tablet_bootstrap.cc:654] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:20.211892 10930 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:20.221091 11065 ts_tablet_manager.cc:1428] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.028s user 0.003s sys 0.003s
I20260809 15:51:20.223467 11065 tablet_bootstrap.cc:492] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:20.224064 11067 tablet_bootstrap.cc:492] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:20.224529 11067 ts_tablet_manager.cc:1397] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.063s user 0.023s sys 0.014s
I20260809 15:51:20.227674 11067 raft_consensus.cc:348] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.228538 11067 raft_consensus.cc:374] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:20.228888 11067 raft_consensus.cc:729] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:20.229918 11065 tablet_bootstrap.cc:654] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:20.229692 11067 consensus_queue.cc:260] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.232821 11067 ts_tablet_manager.cc:1428] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.008s user 0.003s sys 0.003s
I20260809 15:51:20.234035 11067 tablet_bootstrap.cc:492] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:20.237910 11070 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:20.238445 11113 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:20.238791 11070 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } }
I20260809 15:51:20.238961 11113 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.246948 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:20.246755 10737 catalog_manager.cc:5495] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:20.247947 11073 consensus_queue.cc:1035] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260809 15:51:20.248482 10811 leader_election.cc:341] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:20.250509 11149 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:20.251044 11149 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.251573 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:51:20.253720 10809 leader_election.cc:341] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:20.254555 10809 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:20.253840 11149 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:20.255674 11109 raft_consensus.cc:2726] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:20.256299 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:20.257040 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:20.250070 11113 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:20.258159 11016 raft_consensus.cc:2379] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 1: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:51:20.260435 10809 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:20.263149 11109 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:20.275527 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:20.276392 10930 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:20.278239 10811 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:20.271328 11073 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:20.280443 11073 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.273689 11070 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:20.283819 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:20.290720 11016 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:20.284776 11113 raft_consensus.cc:2781] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:20.293766 11072 tablet_bootstrap.cc:492] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:20.294275 11072 ts_tablet_manager.cc:1397] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.104s user 0.014s sys 0.000s
I20260809 15:51:20.293916 11073 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:20.295302 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:51:20.297019 10978 leader_election.cc:341] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:20.297910 11072 raft_consensus.cc:348] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.299081 11072 raft_consensus.cc:374] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:20.299391 11072 raft_consensus.cc:729] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:20.299976 11072 consensus_queue.cc:260] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.293009 11113 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:20.301954 11067 tablet_bootstrap.cc:654] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:20.281756 11109 raft_consensus.cc:2726] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:20.298568 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:20.304574 11072 ts_tablet_manager.cc:1428] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.010s user 0.006s sys 0.000s
I20260809 15:51:20.302626 11113 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:20.310513 10847 raft_consensus.cc:2454] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:20.314854 11072 tablet_bootstrap.cc:492] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:20.318723 10978 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:20.321122 11106 raft_consensus.cc:2781] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:20.321834 11106 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:20.322593 11106 raft_consensus.cc:3037] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:20.330924 11073 consensus_queue.cc:1035] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:20.331843 11065 tablet_bootstrap.cc:492] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:20.332388 11065 ts_tablet_manager.cc:1397] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.109s user 0.027s sys 0.056s
I20260809 15:51:20.335884 11065 raft_consensus.cc:348] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.336536 11065 raft_consensus.cc:374] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:20.336921 11065 raft_consensus.cc:729] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:20.337611 11065 consensus_queue.cc:260] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.339773 11072 tablet_bootstrap.cc:654] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:20.341536 11065 ts_tablet_manager.cc:1428] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.009s user 0.009s sys 0.001s
I20260809 15:51:20.340993 11106 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.343740 11065 tablet_bootstrap.cc:492] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:20.344534 11113 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } }
I20260809 15:51:20.347472 11106 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:20.349149 11113 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:20.350886 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:20.351785 10930 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:20.351866 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:20.353107 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:20.356178 11016 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:20.360905 11067 tablet_bootstrap.cc:492] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:20.361394 11067 ts_tablet_manager.cc:1397] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.128s user 0.009s sys 0.006s
I20260809 15:51:20.365590 11067 raft_consensus.cc:348] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.366324 11067 raft_consensus.cc:374] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:20.366551 11067 raft_consensus.cc:729] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
W20260809 15:51:20.367141 10978 leader_election.cc:341] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:20.367882 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:20.368496 10848 raft_consensus.cc:3037] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:20.370708 11067 consensus_queue.cc:260] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.369174 11109 consensus_queue.cc:1035] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260809 15:51:20.377048 10930 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:20.378356 11072 tablet_bootstrap.cc:492] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:20.378818 11072 ts_tablet_manager.cc:1397] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.064s user 0.016s sys 0.003s
I20260809 15:51:20.370365 11106 consensus_queue.cc:1035] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.012s
I20260809 15:51:20.382840 10811 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:20.386313 11065 tablet_bootstrap.cc:654] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:20.382411 11072 raft_consensus.cc:348] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.387243 11072 raft_consensus.cc:374] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:20.387799 11109 raft_consensus.cc:2781] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:20.388442 11109 raft_consensus.cc:686] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:20.389755 11109 consensus_queue.cc:237] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } }
I20260809 15:51:20.390770 11072 raft_consensus.cc:729] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:20.391726 11072 consensus_queue.cc:260] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.396095 11126 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:20.396909 11126 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.397994 11072 ts_tablet_manager.cc:1428] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.019s user 0.009s sys 0.000s
I20260809 15:51:20.400869 11067 ts_tablet_manager.cc:1428] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.039s user 0.008s sys 0.000s
I20260809 15:51:20.403386 10848 raft_consensus.cc:2454] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:20.406697 11115 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:20.407584 11115 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.408917 11126 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:20.410068 11115 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:20.412163 10978 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:20.415634 11067 tablet_bootstrap.cc:492] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:20.415881 11073 raft_consensus.cc:2781] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:20.416457 11073 raft_consensus.cc:686] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:20.422914 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:20.413775 11016 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:20.436261 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:20.441339 10930 raft_consensus.cc:2379] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 1: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:51:20.443782 10930 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:20.446662 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:20.446538 10978 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:20.448736 11083 consensus_queue.cc:1035] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:20.449002 11113 consensus_queue.cc:1035] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:20.449440 11073 consensus_queue.cc:237] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.449363 11016 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:20.461741 11072 tablet_bootstrap.cc:492] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:20.463138 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:20.460335 11083 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:20.510229 11070 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:20.515278 11070 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.522445 11113 consensus_queue.cc:1035] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:20.465003 11083 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:20.547161 10789 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 15:51:20.556169 11065 tablet_bootstrap.cc:492] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:20.556712 11065 ts_tablet_manager.cc:1397] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.213s user 0.015s sys 0.032s
I20260809 15:51:20.543229 11070 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:20.580472 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:20.578640 11083 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:20.568300 10891 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:20.586351 11018 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:20.613481 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:20.611402 11126 raft_consensus.cc:482] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:20.625672 11126 raft_consensus.cc:504] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.626871 10848 raft_consensus.cc:3037] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:20.634737 11188 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:20.635334 11188 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.639099 11188 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:20.640497 11113 consensus_queue.cc:1035] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:20.642932 10809 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:20.647653 11113 raft_consensus.cc:2781] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:20.647993 11113 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:20.648277 11113 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:51:20.644141 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 78746.1us
I20260809 15:51:20.644850 11106 raft_consensus.cc:2726] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:20.646514 11188 raft_consensus.cc:2726] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 15:51:20.650386 10893 leader_election.cc:341] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:20.650655 11065 raft_consensus.cc:348] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.651304 11065 raft_consensus.cc:374] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:20.651476 11065 raft_consensus.cc:729] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:20.669085 11126 leader_election.cc:288] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:20.674446 11115 consensus_queue.cc:1035] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:20.692739 10737 catalog_manager.cc:5495] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:20.719200 11065 consensus_queue.cc:260] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.723639 11067 tablet_bootstrap.cc:654] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:20.732726 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:20.726688 11106 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:20.735615 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:20.738580 11065 ts_tablet_manager.cc:1428] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.182s user 0.025s sys 0.008s
I20260809 15:51:20.739513 11065 tablet_bootstrap.cc:492] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:20.729153 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:20.742470 11106 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.744602 11106 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:20.745541 11072 tablet_bootstrap.cc:654] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:20.750798 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:20.754227 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:20.755038 10844 raft_consensus.cc:2454] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:20.756757 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:20.758641 10978 leader_election.cc:341] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:20.756696 10735 catalog_manager.cc:5495] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 0 to 1, leader changed from <none> to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:20.770546 11113 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.772898 11115 consensus_queue.cc:1035] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:20.774343 11113 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:20.778368 11109 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:20.778606 11195 consensus_queue.cc:1035] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260809 15:51:20.786207 10893 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:20.786998 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
W20260809 15:51:20.788745 10893 leader_election.cc:341] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:20.789171 11018 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:20.787993 11188 raft_consensus.cc:2781] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:20.791172 11188 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:20.794265 11188 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:20.794641 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:20.790633 10891 leader_election.cc:341] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:20.799654 11070 consensus_queue.cc:1035] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.004s
I20260809 15:51:20.801515 11106 consensus_queue.cc:1035] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.008s
W20260809 15:51:20.805687 10978 leader_election.cc:341] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:20.813199 10978 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:20.814258 11106 raft_consensus.cc:2726] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:20.823817 11067 tablet_bootstrap.cc:492] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:20.825176 11067 ts_tablet_manager.cc:1397] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.410s user 0.013s sys 0.029s
W20260809 15:51:20.820262 10891 leader_election.cc:341] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:20.832787 10891 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:20.849470 11067 raft_consensus.cc:348] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.850575 11067 raft_consensus.cc:374] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:20.850833 11067 raft_consensus.cc:729] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:20.851347 11067 consensus_queue.cc:260] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.849491 11083 raft_consensus.cc:2726] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:20.856289 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:20.857172 11017 raft_consensus.cc:2454] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
W20260809 15:51:20.858701 10811 leader_election.cc:341] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:20.860677 11067 ts_tablet_manager.cc:1428] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.035s user 0.006s sys 0.002s
I20260809 15:51:20.860016 11188 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.861999 11067 tablet_bootstrap.cc:492] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:20.866366 11113 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:20.874675 11065 tablet_bootstrap.cc:654] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:20.865487 11018 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:20.878072 10809 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:20.880084 11113 raft_consensus.cc:2781] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:20.882671 11113 raft_consensus.cc:686] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:20.889729 11113 consensus_queue.cc:237] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.894038 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:20.895081 10848 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:20.900416 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
W20260809 15:51:20.906494 10811 leader_election.cc:341] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
W20260809 15:51:20.915107 10978 consensus_peers.cc:487] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa -> Peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Couldn't send request to peer 20730d75ad954a82b126148f8152a9f3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:20.909006 10891 leader_election.cc:302] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:20.915388 11188 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:20.936493 11188 raft_consensus.cc:2781] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:20.936920 11188 raft_consensus.cc:482] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:20.937352 11188 raft_consensus.cc:3037] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:20.955667 11178 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:20.956334 11178 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:20.957762 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 48935.7us
I20260809 15:51:20.958602 11067 tablet_bootstrap.cc:654] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:20.967995 11178 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:20.969882 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:20.958271 11188 raft_consensus.cc:504] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.975104 11072 tablet_bootstrap.cc:492] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:20.975641 11072 ts_tablet_manager.cc:1397] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.514s user 0.034s sys 0.054s
I20260809 15:51:20.957196 10848 raft_consensus.cc:2454] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
W20260809 15:51:20.976330 10978 leader_election.cc:341] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:20.977854 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:51:20.979460 10893 leader_election.cc:341] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:20.979554 11072 raft_consensus.cc:348] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.980350 11072 raft_consensus.cc:374] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:20.980687 11072 raft_consensus.cc:729] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:20.982803 11188 leader_election.cc:288] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:20.981346 11072 consensus_queue.cc:260] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.984333 10893 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:20.984530 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:20.986286 11072 ts_tablet_manager.cc:1428] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260809 15:51:20.986021 11083 raft_consensus.cc:2781] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election won for term 1
W20260809 15:51:20.986545 10978 leader_election.cc:341] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:20.986882 11083 raft_consensus.cc:686] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:20.987138 11072 tablet_bootstrap.cc:492] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:20.987124 10978 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:20.988155 11083 consensus_queue.cc:237] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:20.989488 11178 raft_consensus.cc:2726] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:20.984337 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
W20260809 15:51:20.996727 10891 leader_election.cc:341] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:20.997486 11065 tablet_bootstrap.cc:492] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:20.997946 11065 ts_tablet_manager.cc:1397] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.259s user 0.024s sys 0.025s
I20260809 15:51:21.000667 11178 consensus_queue.cc:1035] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:21.005702 10737 catalog_manager.cc:5495] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 0 to 1, leader changed from <none> to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:21.008648 11065 raft_consensus.cc:348] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.010588 11065 raft_consensus.cc:374] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:21.010895 11065 raft_consensus.cc:729] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:21.011569 11065 consensus_queue.cc:260] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.013770 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:21.014269 11065 ts_tablet_manager.cc:1428] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.016s user 0.005s sys 0.002s
I20260809 15:51:21.014626 11017 raft_consensus.cc:3037] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:21.015404 11065 tablet_bootstrap.cc:492] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:21.024703 11067 tablet_bootstrap.cc:492] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:21.025223 11067 ts_tablet_manager.cc:1397] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.164s user 0.016s sys 0.007s
I20260809 15:51:21.030287 11113 consensus_queue.cc:1035] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:21.037408 11072 tablet_bootstrap.cc:654] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:21.060580 11067 raft_consensus.cc:348] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.061540 11067 raft_consensus.cc:374] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:21.061892 11067 raft_consensus.cc:729] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:21.062788 11067 consensus_queue.cc:260] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.060336 11017 raft_consensus.cc:2454] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:21.070209 10891 leader_election.cc:302] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:21.071971 11083 raft_consensus.cc:2781] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:21.072306 11065 tablet_bootstrap.cc:654] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:21.072454 11083 raft_consensus.cc:686] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:21.073755 11083 consensus_queue.cc:237] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.080938 11067 ts_tablet_manager.cc:1428] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.055s user 0.008s sys 0.002s
I20260809 15:51:21.090322 11072 tablet_bootstrap.cc:492] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:21.090793 11072 ts_tablet_manager.cc:1397] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.104s user 0.013s sys 0.013s
I20260809 15:51:21.093914 11067 tablet_bootstrap.cc:492] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:21.098747 10734 catalog_manager.cc:5495] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:21.111001 11072 raft_consensus.cc:348] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.111740 11072 raft_consensus.cc:374] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:21.112041 11072 raft_consensus.cc:729] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:21.113314 11072 consensus_queue.cc:260] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.117259 11072 ts_tablet_manager.cc:1428] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.026s user 0.006s sys 0.000s
I20260809 15:51:21.118261 11072 tablet_bootstrap.cc:492] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:21.127041 11067 tablet_bootstrap.cc:654] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:21.127089 11072 tablet_bootstrap.cc:654] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:21.129980 10737 catalog_manager.cc:5495] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:21.145125 11065 tablet_bootstrap.cc:492] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:21.145625 11065 ts_tablet_manager.cc:1397] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.130s user 0.020s sys 0.014s
I20260809 15:51:21.146318 11067 tablet_bootstrap.cc:492] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:21.146996 11067 ts_tablet_manager.cc:1397] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.063s user 0.012s sys 0.017s
I20260809 15:51:21.149696 11067 raft_consensus.cc:348] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.150702 11067 raft_consensus.cc:374] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:21.151168 11067 raft_consensus.cc:729] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:21.151423 11072 tablet_bootstrap.cc:492] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:21.152417 11072 ts_tablet_manager.cc:1397] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.034s user 0.003s sys 0.025s
I20260809 15:51:21.152042 11067 consensus_queue.cc:260] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.155151 11067 ts_tablet_manager.cc:1428] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260809 15:51:21.156399 11067 tablet_bootstrap.cc:492] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:21.156474 11065 raft_consensus.cc:348] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.157254 11065 raft_consensus.cc:374] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:21.157550 11065 raft_consensus.cc:729] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:21.158272 11065 consensus_queue.cc:260] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.160805 11065 ts_tablet_manager.cc:1428] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.015s user 0.003s sys 0.003s
I20260809 15:51:21.160475 11072 raft_consensus.cc:348] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.161505 11072 raft_consensus.cc:374] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:21.161893 11072 raft_consensus.cc:729] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:21.162730 11072 consensus_queue.cc:260] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.165329 11072 ts_tablet_manager.cc:1428] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20260809 15:51:21.168149 11067 tablet_bootstrap.cc:654] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:21.169708 11072 tablet_bootstrap.cc:492] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:21.184423 11065 tablet_bootstrap.cc:492] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:21.202015 11065 tablet_bootstrap.cc:654] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:21.202157 11067 tablet_bootstrap.cc:492] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:21.202639 11067 ts_tablet_manager.cc:1397] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.046s user 0.006s sys 0.017s
I20260809 15:51:21.203640 11072 tablet_bootstrap.cc:654] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:21.205700 11067 raft_consensus.cc:348] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.206502 11067 raft_consensus.cc:374] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:21.206892 11067 raft_consensus.cc:729] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:21.207959 11067 consensus_queue.cc:260] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.209967 11067 ts_tablet_manager.cc:1428] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260809 15:51:21.211102 11067 tablet_bootstrap.cc:492] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:21.219318 11067 tablet_bootstrap.cc:654] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:21.239322 11065 tablet_bootstrap.cc:492] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:21.239885 11065 ts_tablet_manager.cc:1397] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.056s user 0.013s sys 0.028s
I20260809 15:51:21.242615 11072 tablet_bootstrap.cc:492] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:21.243086 11072 ts_tablet_manager.cc:1397] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.074s user 0.016s sys 0.000s
I20260809 15:51:21.243039 11065 raft_consensus.cc:348] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.243911 11065 raft_consensus.cc:374] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:21.244266 11065 raft_consensus.cc:729] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:21.245589 11065 consensus_queue.cc:260] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.247354 11072 raft_consensus.cc:348] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.247977 11072 raft_consensus.cc:374] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:21.248195 11072 raft_consensus.cc:729] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:21.248659 11067 tablet_bootstrap.cc:492] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:21.249063 11067 ts_tablet_manager.cc:1397] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.038s user 0.008s sys 0.011s
I20260809 15:51:21.248817 11072 consensus_queue.cc:260] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.250909 11072 ts_tablet_manager.cc:1428] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 15:51:21.251968 11072 tablet_bootstrap.cc:492] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:21.255564 11065 ts_tablet_manager.cc:1428] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260809 15:51:21.257764 11072 tablet_bootstrap.cc:654] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:21.259713 11065 tablet_bootstrap.cc:492] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:21.261788 11067 raft_consensus.cc:348] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.264681 11178 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:21.265151 11178 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.262650 11067 raft_consensus.cc:374] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:21.267760 11067 raft_consensus.cc:729] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:21.268620 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:21.268412 11067 consensus_queue.cc:260] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.269248 10848 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:21.270550 10978 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:21.272028 11073 raft_consensus.cc:2781] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:21.272370 11073 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:21.272733 11073 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:21.274433 11113 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:21.275161 11113 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.277606 11113 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:21.279214 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:21.280071 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:21.280755 11186 raft_consensus.cc:482] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:51:21.281149 10811 leader_election.cc:341] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:21.281282 11186 raft_consensus.cc:504] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.274556 11067 ts_tablet_manager.cc:1428] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.025s user 0.006s sys 0.001s
I20260809 15:51:21.280776 11017 raft_consensus.cc:2454] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:21.283533 11178 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:21.284022 11067 tablet_bootstrap.cc:492] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:21.285701 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:51:21.288592 10978 leader_election.cc:341] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:21.291986 11186 leader_election.cc:288] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:21.291285 10809 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:21.295694 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:21.296502 11113 raft_consensus.cc:2781] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:21.296803 11113 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:21.297240 11113 raft_consensus.cc:3037] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:51:21.297284 10978 leader_election.cc:341] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:21.300880 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:21.302354 11065 tablet_bootstrap.cc:654] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
W20260809 15:51:21.302632 10978 leader_election.cc:341] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:21.303123 10978 leader_election.cc:302] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:21.304138 11186 raft_consensus.cc:2726] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:21.311439 11073 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.312436 11067 tablet_bootstrap.cc:654] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:21.314503 11072 tablet_bootstrap.cc:492] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:21.314994 11072 ts_tablet_manager.cc:1397] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.063s user 0.011s sys 0.013s
I20260809 15:51:21.318257 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:21.318305 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:21.318609 11072 raft_consensus.cc:348] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.319456 11072 raft_consensus.cc:374] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:21.319731 11072 raft_consensus.cc:729] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
W20260809 15:51:21.320168 10978 leader_election.cc:341] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:21.319130 10848 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:21.320586 11072 consensus_queue.cc:260] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.330055 10848 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:21.331902 11072 ts_tablet_manager.cc:1428] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.017s user 0.006s sys 0.000s
I20260809 15:51:21.320961 11073 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:21.333920 11072 tablet_bootstrap.cc:492] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:21.336026 11109 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:21.336761 11109 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.338249 11195 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:21.338886 11195 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.341967 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:21.342885 10929 raft_consensus.cc:2454] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:21.343978 11195 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:21.343986 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:21.353219 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:21.354008 11015 raft_consensus.cc:2379] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 1: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:51:21.360363 11067 tablet_bootstrap.cc:492] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:21.359824 10978 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:21.360930 11067 ts_tablet_manager.cc:1397] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.077s user 0.010s sys 0.009s
I20260809 15:51:21.361373 11073 raft_consensus.cc:2781] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:21.363735 10891 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:21.364254 11073 raft_consensus.cc:686] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:21.361718 10811 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:21.367017 11109 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:21.367555 10929 raft_consensus.cc:3037] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:21.369408 11067 raft_consensus.cc:348] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.366365 11070 raft_consensus.cc:2781] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:21.370724 11195 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:21.372124 11070 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:21.372610 11070 raft_consensus.cc:3037] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:21.374328 11067 raft_consensus.cc:374] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:21.374643 11067 raft_consensus.cc:729] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:21.375257 11067 consensus_queue.cc:260] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.377703 11188 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:21.378162 11188 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.380874 11188 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:21.366039 11073 consensus_queue.cc:237] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.371500 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:21.382493 11067 ts_tablet_manager.cc:1428] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.021s user 0.003s sys 0.006s
I20260809 15:51:21.347333 11113 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.384567 11015 raft_consensus.cc:2454] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:21.389030 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:21.390825 10893 leader_election.cc:341] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:21.392949 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:21.393700 11017 raft_consensus.cc:2454] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:21.399823 11067 tablet_bootstrap.cc:492] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:21.402040 11113 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:21.401662 10891 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:21.405007 11126 raft_consensus.cc:2781] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:21.393710 11070 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.406738 11126 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:21.407697 11126 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:21.410673 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:21.411375 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
W20260809 15:51:21.411931 10811 leader_election.cc:341] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:21.412089 11015 raft_consensus.cc:3037] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:21.413964 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:21.414343 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:21.415015 11070 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:21.415225 11017 raft_consensus.cc:3037] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:21.414613 10931 raft_consensus.cc:3037] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:21.425239 11015 raft_consensus.cc:2454] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:21.426676 10809 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:21.427816 11070 raft_consensus.cc:2781] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:21.428956 11195 raft_consensus.cc:2726] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:21.432436 11067 tablet_bootstrap.cc:654] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:21.430892 11070 raft_consensus.cc:686] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:21.437249 11186 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:21.437834 11186 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.438557 11065 tablet_bootstrap.cc:492] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:21.438957 11065 ts_tablet_manager.cc:1397] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.179s user 0.026s sys 0.025s
I20260809 15:51:21.432904 11113 raft_consensus.cc:482] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:21.439962 11113 raft_consensus.cc:504] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.440838 11186 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:21.442893 11113 leader_election.cc:288] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:21.443583 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:21.453410 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:51:21.455207 10978 leader_election.cc:341] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:21.459208 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:21.458889 11070 consensus_queue.cc:237] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.460319 11015 raft_consensus.cc:2379] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 1: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:51:21.460879 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:21.450011 11126 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.462776 10811 leader_election.cc:302] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:21.446498 11113 consensus_queue.cc:1035] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.018s
W20260809 15:51:21.468094 10978 leader_election.cc:341] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:21.468940 10735 catalog_manager.cc:5495] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:21.471690 11072 tablet_bootstrap.cc:654] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:21.472236 11065 raft_consensus.cc:348] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.473256 11065 raft_consensus.cc:374] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:21.473582 11065 raft_consensus.cc:729] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:21.475492 11065 consensus_queue.cc:260] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.478781 11126 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:21.468572 10978 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:21.480297 11186 raft_consensus.cc:2726] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:21.480428 11017 raft_consensus.cc:2454] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:21.481172 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:21.482254 10931 raft_consensus.cc:2454] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:21.482524 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:21.483299 11017 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:21.496084 11109 raft_consensus.cc:3037] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:21.498590 11126 consensus_queue.cc:1035] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 15:51:21.500278 10893 leader_election.cc:341] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:21.500898 10809 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:21.507846 11236 raft_consensus.cc:2781] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:21.508442 11236 raft_consensus.cc:686] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:21.515588 11065 ts_tablet_manager.cc:1428] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.076s user 0.009s sys 0.005s
I20260809 15:51:21.516749 11065 tablet_bootstrap.cc:492] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:21.509634 11236 consensus_queue.cc:237] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.513720 11109 raft_consensus.cc:2726] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260809 15:51:21.536167 10891 consensus_peers.cc:487] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 -> Peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Couldn't send request to peer d74944fa9fa14ec78b0640ea709ff3aa. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260809 15:51:21.536929 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 74241.7us
I20260809 15:51:21.545447 11017 raft_consensus.cc:2454] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:21.546656 10891 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:21.548470 11126 raft_consensus.cc:2781] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:21.552513 11126 raft_consensus.cc:686] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:21.553537 11126 consensus_queue.cc:237] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.579824 10735 catalog_manager.cc:5495] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 0 to 1, leader changed from <none> to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
W20260809 15:51:21.569375 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 68597.8us
I20260809 15:51:21.592015 11231 raft_consensus.cc:482] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:21.592901 11231 raft_consensus.cc:504] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.605654 11065 tablet_bootstrap.cc:654] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:21.616271 11072 tablet_bootstrap.cc:492] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:21.616981 11072 ts_tablet_manager.cc:1397] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.283s user 0.015s sys 0.014s
I20260809 15:51:21.617949 11067 tablet_bootstrap.cc:492] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:21.618649 11195 consensus_queue.cc:1035] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:21.619208 11083 consensus_queue.cc:1035] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:21.622246 11067 ts_tablet_manager.cc:1397] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.223s user 0.037s sys 0.039s
I20260809 15:51:21.622466 11231 leader_election.cc:288] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:21.622675 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:21.623514 11016 raft_consensus.cc:2454] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:21.625232 11067 raft_consensus.cc:348] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.626163 11067 raft_consensus.cc:374] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:21.626466 11067 raft_consensus.cc:729] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:21.627241 11067 consensus_queue.cc:260] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.633421 11109 consensus_queue.cc:1035] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:21.636807 10891 leader_election.cc:302] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:21.638460 11126 raft_consensus.cc:2781] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:21.639194 11067 ts_tablet_manager.cc:1428] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.017s user 0.008s sys 0.000s
I20260809 15:51:21.638864 11126 raft_consensus.cc:482] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:21.640887 11126 raft_consensus.cc:3037] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:21.665215 11109 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:21.681455 11109 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.683357 11109 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:21.684398 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:21.693033 10893 leader_election.cc:341] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:21.698380 11072 raft_consensus.cc:348] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.699357 11072 raft_consensus.cc:374] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:21.699643 11072 raft_consensus.cc:729] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:21.700821 11067 tablet_bootstrap.cc:492] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:21.703934 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:21.684195 10733 catalog_manager.cc:5495] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 0 to 1, leader changed from <none> to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:21.701233 11072 consensus_queue.cc:260] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:21.712704 10811 leader_election.cc:341] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:21.714844 10736 catalog_manager.cc:5495] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:21.716331 11072 ts_tablet_manager.cc:1428] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.099s user 0.004s sys 0.004s
I20260809 15:51:21.718030 11072 tablet_bootstrap.cc:492] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:21.734974 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:21.735924 11016 raft_consensus.cc:2454] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:21.744858 10809 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:21.745618 11070 raft_consensus.cc:2781] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:21.747022 11072 tablet_bootstrap.cc:654] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:21.747077 11070 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:21.747581 11070 raft_consensus.cc:3037] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:21.753669 11067 tablet_bootstrap.cc:654] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
W20260809 15:51:21.757076 10789 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 15:51:21.776110 11065 tablet_bootstrap.cc:492] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:21.776599 11065 ts_tablet_manager.cc:1397] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.260s user 0.017s sys 0.025s
I20260809 15:51:21.780649 11065 raft_consensus.cc:348] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.781311 11065 raft_consensus.cc:374] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:21.781555 11065 raft_consensus.cc:729] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:21.771862 11126 raft_consensus.cc:504] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.786299 11073 consensus_queue.cc:1035] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:21.785823 11065 consensus_queue.cc:260] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.793671 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:21.812361 11070 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.843079 11126 leader_election.cc:288] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:21.847424 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:21.847949 11017 raft_consensus.cc:3037] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:21.852118 11065 ts_tablet_manager.cc:1428] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.075s user 0.026s sys 0.030s
I20260809 15:51:21.842351 10844 raft_consensus.cc:3037] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:21.855774 11065 tablet_bootstrap.cc:492] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:21.859246 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:21.861743 11070 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:51:21.862476 10811 leader_election.cc:341] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:21.864629 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:21.865782 11015 raft_consensus.cc:3037] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:21.873417 11067 tablet_bootstrap.cc:492] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:21.873826 11067 ts_tablet_manager.cc:1397] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.232s user 0.021s sys 0.024s
I20260809 15:51:21.882886 11067 raft_consensus.cc:348] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.883190 10844 raft_consensus.cc:2454] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
W20260809 15:51:21.884720 10978 consensus_peers.cc:487] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa -> Peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Couldn't send request to peer 20730d75ad954a82b126148f8152a9f3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:21.883731 11067 raft_consensus.cc:374] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:21.891994 11067 raft_consensus.cc:729] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:21.892423 11015 raft_consensus.cc:2454] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:21.892813 11067 consensus_queue.cc:260] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.893651 10809 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:21.894198 11017 raft_consensus.cc:2454] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:21.895454 11067 ts_tablet_manager.cc:1428] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.021s user 0.006s sys 0.000s
I20260809 15:51:21.895529 10891 leader_election.cc:302] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:21.896350 11067 tablet_bootstrap.cc:492] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:21.896831 11070 raft_consensus.cc:2781] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:21.897367 11070 raft_consensus.cc:686] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:21.896701 11231 raft_consensus.cc:2781] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:21.898049 11231 raft_consensus.cc:686] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:21.898509 11072 tablet_bootstrap.cc:492] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:21.898881 11231 consensus_queue.cc:237] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.902627 11065 tablet_bootstrap.cc:654] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:21.902064 11070 consensus_queue.cc:237] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.905171 11186 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:21.898942 11072 ts_tablet_manager.cc:1397] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.181s user 0.011s sys 0.005s
I20260809 15:51:21.905695 11186 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.908630 11186 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:21.909024 11072 raft_consensus.cc:348] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.909734 11072 raft_consensus.cc:374] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:21.910066 11072 raft_consensus.cc:729] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:21.910755 11072 consensus_queue.cc:260] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.911013 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:51:21.912858 10978 leader_election.cc:341] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:21.914670 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:21.913084 11072 ts_tablet_manager.cc:1428] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
W20260809 15:51:21.915902 10978 leader_election.cc:341] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:21.916411 10978 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:21.917331 11258 raft_consensus.cc:2726] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:21.923442 11067 tablet_bootstrap.cc:654] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:21.926903 11072 tablet_bootstrap.cc:492] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:21.942782 11065 tablet_bootstrap.cc:492] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:21.953819 11065 ts_tablet_manager.cc:1397] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.098s user 0.013s sys 0.013s
I20260809 15:51:21.954645 11258 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:21.955320 11072 tablet_bootstrap.cc:654] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:21.963505 11070 consensus_queue.cc:1035] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:21.956629 11065 raft_consensus.cc:348] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.964574 11065 raft_consensus.cc:374] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:21.953294 10734 catalog_manager.cc:5495] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:21.976022 11109 consensus_queue.cc:1035] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:21.964839 11065 raft_consensus.cc:729] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:21.981842 11065 consensus_queue.cc:260] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.963938 11258 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:21.986929 11065 ts_tablet_manager.cc:1428] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.033s user 0.006s sys 0.000s
I20260809 15:51:21.988487 11065 tablet_bootstrap.cc:492] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:21.988245 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:21.991241 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:21.978976 10737 catalog_manager.cc:5495] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 0 to 1, leader changed from <none> to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
W20260809 15:51:21.992936 10978 leader_election.cc:341] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
W20260809 15:51:21.993669 10978 leader_election.cc:341] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:21.994102 10978 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:21.995172 11073 raft_consensus.cc:2726] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:22.000520 11258 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:22.006117 11065 tablet_bootstrap.cc:654] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.006557 11072 tablet_bootstrap.cc:492] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:22.011389 11072 ts_tablet_manager.cc:1397] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.085s user 0.017s sys 0.029s
I20260809 15:51:22.017808 11072 raft_consensus.cc:348] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.018587 11072 raft_consensus.cc:374] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.019004 11072 raft_consensus.cc:729] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.019899 11072 consensus_queue.cc:260] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.021179 11067 tablet_bootstrap.cc:492] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:22.021657 11067 ts_tablet_manager.cc:1397] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.125s user 0.015s sys 0.005s
I20260809 15:51:22.023744 11072 ts_tablet_manager.cc:1428] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.012s user 0.008s sys 0.000s
I20260809 15:51:22.024926 11072 tablet_bootstrap.cc:492] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:22.024577 11067 raft_consensus.cc:348] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.030581 11067 raft_consensus.cc:374] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
W20260809 15:51:22.037303 10811 consensus_peers.cc:487] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 -> Peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Couldn't send request to peer 20730d75ad954a82b126148f8152a9f3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:22.030851 11067 raft_consensus.cc:729] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.043026 11067 consensus_queue.cc:260] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.045059 11067 ts_tablet_manager.cc:1428] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.023s user 0.003s sys 0.002s
I20260809 15:51:22.045835 11067 tablet_bootstrap.cc:492] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:22.050937 11109 consensus_queue.cc:1035] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:22.051610 11258 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:22.052234 11258 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.056447 11258 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:22.057271 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:22.062951 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:22.065176 11065 tablet_bootstrap.cc:492] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:22.065294 11017 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:22.065724 11065 ts_tablet_manager.cc:1397] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.077s user 0.018s sys 0.021s
I20260809 15:51:22.067015 11072 tablet_bootstrap.cc:654] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.069002 10978 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:22.084447 11067 tablet_bootstrap.cc:654] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.090134 11065 raft_consensus.cc:348] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.090934 11065 raft_consensus.cc:374] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.091251 11065 raft_consensus.cc:729] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.092129 11065 consensus_queue.cc:260] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.094641 11065 ts_tablet_manager.cc:1428] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.029s user 0.014s sys 0.006s
I20260809 15:51:22.095974 11065 tablet_bootstrap.cc:492] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:22.102875 11065 tablet_bootstrap.cc:654] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.105837 11258 raft_consensus.cc:2726] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:22.126482 11067 tablet_bootstrap.cc:492] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:22.126941 11067 ts_tablet_manager.cc:1397] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.081s user 0.010s sys 0.008s
I20260809 15:51:22.128649 11126 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:22.129282 11126 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.131608 11126 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:22.133671 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:22.134408 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:22.134713 10844 raft_consensus.cc:2454] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:22.135150 11015 raft_consensus.cc:2454] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:22.135928 10893 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:22.137106 11126 raft_consensus.cc:2781] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:22.137632 11126 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:22.138417 11126 raft_consensus.cc:3037] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:22.140282 11067 raft_consensus.cc:348] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.141170 11067 raft_consensus.cc:374] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.141552 11067 raft_consensus.cc:729] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.142377 11067 consensus_queue.cc:260] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.145754 11067 ts_tablet_manager.cc:1428] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.019s user 0.007s sys 0.000s
I20260809 15:51:22.147567 11067 tablet_bootstrap.cc:492] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:22.152546 11072 tablet_bootstrap.cc:492] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:22.153167 11072 ts_tablet_manager.cc:1397] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.128s user 0.004s sys 0.016s
I20260809 15:51:22.156627 11072 raft_consensus.cc:348] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.168648 11065 tablet_bootstrap.cc:492] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:22.169054 11072 raft_consensus.cc:374] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.169188 11065 ts_tablet_manager.cc:1397] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.073s user 0.015s sys 0.020s
I20260809 15:51:22.169080 11067 tablet_bootstrap.cc:654] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.169579 11072 raft_consensus.cc:729] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.171782 11072 consensus_queue.cc:260] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.172262 11065 raft_consensus.cc:348] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.173053 11065 raft_consensus.cc:374] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.173313 11065 raft_consensus.cc:729] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.174029 11065 consensus_queue.cc:260] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.176780 11065 ts_tablet_manager.cc:1428] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.007s user 0.002s sys 0.003s
I20260809 15:51:22.177348 11231 consensus_queue.cc:1035] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:22.180909 11065 tablet_bootstrap.cc:492] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:22.187290 11072 ts_tablet_manager.cc:1428] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.034s user 0.002s sys 0.005s
I20260809 15:51:22.188279 11072 tablet_bootstrap.cc:492] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:22.193305 11065 tablet_bootstrap.cc:654] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
W20260809 15:51:22.195216 10893 consensus_peers.cc:487] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 -> Peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Couldn't send request to peer a53441872c0c416b8e2f26f52399f536. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:22.207572 11072 tablet_bootstrap.cc:654] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.210116 11126 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.214177 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:22.214932 10844 raft_consensus.cc:3037] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:22.233355 10844 raft_consensus.cc:2454] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:22.234184 11065 tablet_bootstrap.cc:492] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:22.234699 11065 ts_tablet_manager.cc:1397] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.057s user 0.012s sys 0.010s
I20260809 15:51:22.235056 10893 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:22.236613 11188 raft_consensus.cc:2781] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:22.237397 11188 raft_consensus.cc:686] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:22.238495 11188 consensus_queue.cc:237] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.240504 11065 raft_consensus.cc:348] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.241958 11065 raft_consensus.cc:374] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.242334 11065 raft_consensus.cc:729] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.243132 11065 consensus_queue.cc:260] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.245848 11065 ts_tablet_manager.cc:1428] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260809 15:51:22.249363 11072 tablet_bootstrap.cc:492] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:22.249862 11072 ts_tablet_manager.cc:1397] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.062s user 0.015s sys 0.000s
I20260809 15:51:22.253366 11065 tablet_bootstrap.cc:492] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:22.262853 11072 raft_consensus.cc:348] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.263686 11072 raft_consensus.cc:374] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.264047 11072 raft_consensus.cc:729] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.264628 11072 consensus_queue.cc:260] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.266136 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:22.266683 11015 raft_consensus.cc:3037] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:22.266937 11072 ts_tablet_manager.cc:1428] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.017s user 0.005s sys 0.002s
I20260809 15:51:22.267768 11072 tablet_bootstrap.cc:492] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:22.271328 10927 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:22.271845 11065 tablet_bootstrap.cc:654] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.263686 11126 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:22.284955 11126 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:22.285564 11126 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.285238 10736 catalog_manager.cc:5495] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:22.290556 11070 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:22.286846 11015 raft_consensus.cc:2454] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:22.291194 11070 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.294860 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:22.296618 11067 tablet_bootstrap.cc:492] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
W20260809 15:51:22.296851 10811 consensus_peers.cc:487] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 -> Peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Couldn't send request to peer 20730d75ad954a82b126148f8152a9f3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260809 15:51:22.297775 10811 leader_election.cc:341] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:22.298758 11070 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:22.298494 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:22.300239 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:22.300429 10844 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:22.297086 11067 ts_tablet_manager.cc:1397] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.150s user 0.033s sys 0.084s
I20260809 15:51:22.302280 10893 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:22.303781 11188 raft_consensus.cc:2781] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:22.303984 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:22.304175 11188 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:22.304865 11188 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:51:22.306097 10891 leader_election.cc:341] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
W20260809 15:51:22.306910 10809 leader_election.cc:341] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:22.307521 10809 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:22.308935 11070 raft_consensus.cc:2726] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:22.312207 11126 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:22.315011 11067 raft_consensus.cc:348] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.315732 11067 raft_consensus.cc:374] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.316056 11067 raft_consensus.cc:729] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.316716 11067 consensus_queue.cc:260] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.319180 11067 ts_tablet_manager.cc:1428] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.017s user 0.001s sys 0.005s
I20260809 15:51:22.320187 11067 tablet_bootstrap.cc:492] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:22.340571 11188 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.344460 11072 tablet_bootstrap.cc:654] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.347290 11126 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:22.347935 11126 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.349141 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:22.350333 10844 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:22.353657 11067 tablet_bootstrap.cc:654] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.355432 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:22.354609 11126 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:22.358811 11126 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:22.360083 11126 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:22.360488 10893 leader_election.cc:341] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:22.360522 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:22.360755 11083 consensus_queue.cc:1035] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260809 15:51:22.362833 10891 leader_election.cc:341] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:22.358724 11188 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:22.365713 11065 tablet_bootstrap.cc:492] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:22.366215 11065 ts_tablet_manager.cc:1397] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.113s user 0.010s sys 0.016s
I20260809 15:51:22.371345 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:22.371145 11065 raft_consensus.cc:348] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.371897 11065 raft_consensus.cc:374] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.372001 11016 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:22.372166 11065 raft_consensus.cc:729] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.372584 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:22.372787 11065 consensus_queue.cc:260] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.367434 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:22.377262 11065 ts_tablet_manager.cc:1428] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.011s user 0.006s sys 0.000s
I20260809 15:51:22.375634 10891 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:22.378405 11234 consensus_queue.cc:1035] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:22.378424 11065 tablet_bootstrap.cc:492] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
W20260809 15:51:22.379374 10891 leader_election.cc:341] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:22.379782 11083 raft_consensus.cc:2781] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:22.374277 11126 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:51:22.381140 10893 leader_election.cc:341] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:22.373589 10844 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:22.380277 11083 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:22.383489 11083 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:22.380256 10891 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:22.386394 11234 raft_consensus.cc:482] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:22.386884 11234 raft_consensus.cc:504] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.387652 11231 raft_consensus.cc:2726] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:22.389617 11234 leader_election.cc:288] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:22.399690 11072 tablet_bootstrap.cc:492] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:22.400389 11072 ts_tablet_manager.cc:1397] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.133s user 0.010s sys 0.022s
I20260809 15:51:22.400615 11067 tablet_bootstrap.cc:492] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:22.401161 11067 ts_tablet_manager.cc:1397] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.081s user 0.020s sys 0.025s
I20260809 15:51:22.396080 10893 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:22.403759 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:22.405802 11067 raft_consensus.cc:348] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.406564 11067 raft_consensus.cc:374] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.406847 11067 raft_consensus.cc:729] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.407400 11067 consensus_queue.cc:260] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.408589 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:51:22.408846 10809 leader_election.cc:341] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:22.409911 11067 ts_tablet_manager.cc:1428] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
W20260809 15:51:22.410341 10811 leader_election.cc:341] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:22.410862 11067 tablet_bootstrap.cc:492] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:22.410835 10811 leader_election.cc:302] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:22.411300 11072 raft_consensus.cc:348] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.411985 11072 raft_consensus.cc:374] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.412223 11072 raft_consensus.cc:729] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.412921 11231 raft_consensus.cc:2781] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:22.412822 11072 consensus_queue.cc:260] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.413522 11231 raft_consensus.cc:686] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:22.414016 11234 raft_consensus.cc:2726] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:22.417618 11072 ts_tablet_manager.cc:1428] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.017s user 0.005s sys 0.001s
I20260809 15:51:22.419550 11072 tablet_bootstrap.cc:492] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:22.436061 11231 consensus_queue.cc:237] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.448256 11126 consensus_queue.cc:1035] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:22.456516 11258 raft_consensus.cc:482] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:22.457311 11258 raft_consensus.cc:504] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.461211 11067 tablet_bootstrap.cc:654] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.466432 11065 tablet_bootstrap.cc:654] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.478302 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:22.482709 10978 leader_election.cc:341] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:22.483109 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:22.471086 11258 leader_election.cc:288] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:22.489244 11072 tablet_bootstrap.cc:654] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.488974 10733 catalog_manager.cc:5495] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
W20260809 15:51:22.494244 10978 leader_election.cc:341] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:22.494867 10978 leader_election.cc:302] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:22.496789 11258 raft_consensus.cc:2726] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:22.495885 11083 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.516296 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:22.516580 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:22.517202 11017 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:51:22.518668 10893 leader_election.cc:341] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:22.511382 11083 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:22.539887 11065 tablet_bootstrap.cc:492] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:22.540318 11065 ts_tablet_manager.cc:1397] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.162s user 0.015s sys 0.009s
I20260809 15:51:22.551221 11017 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:22.553030 10891 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:22.555094 11083 raft_consensus.cc:2781] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:22.556844 11083 raft_consensus.cc:686] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:22.558491 11083 consensus_queue.cc:237] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.562664 11072 tablet_bootstrap.cc:492] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:22.563150 11072 ts_tablet_manager.cc:1397] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.144s user 0.022s sys 0.024s
I20260809 15:51:22.563512 11065 raft_consensus.cc:348] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.564332 11065 raft_consensus.cc:374] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.564735 11065 raft_consensus.cc:729] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.564716 11067 tablet_bootstrap.cc:492] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:22.565464 11067 ts_tablet_manager.cc:1397] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.155s user 0.008s sys 0.018s
I20260809 15:51:22.566191 11065 consensus_queue.cc:260] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.567072 11072 raft_consensus.cc:348] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.567710 11072 raft_consensus.cc:374] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.567972 11072 raft_consensus.cc:729] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.568629 11065 ts_tablet_manager.cc:1428] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.028s user 0.007s sys 0.000s
I20260809 15:51:22.568429 11067 raft_consensus.cc:348] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.568526 11072 consensus_queue.cc:260] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.570281 11065 tablet_bootstrap.cc:492] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:22.570799 11072 ts_tablet_manager.cc:1428] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260809 15:51:22.570365 11067 raft_consensus.cc:374] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.571167 11067 raft_consensus.cc:729] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.571813 11072 tablet_bootstrap.cc:492] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:22.571830 11067 consensus_queue.cc:260] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.574927 11067 ts_tablet_manager.cc:1428] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.009s user 0.004s sys 0.003s
I20260809 15:51:22.578545 11067 tablet_bootstrap.cc:492] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:22.585305 11065 tablet_bootstrap.cc:654] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.592237 11072 tablet_bootstrap.cc:654] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.594178 11067 tablet_bootstrap.cc:654] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.610263 10848 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:22.624805 11065 tablet_bootstrap.cc:492] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:22.625296 11065 ts_tablet_manager.cc:1397] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.055s user 0.012s sys 0.004s
I20260809 15:51:22.627758 11067 tablet_bootstrap.cc:492] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:22.628263 11067 ts_tablet_manager.cc:1397] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.050s user 0.012s sys 0.011s
I20260809 15:51:22.627832 11065 raft_consensus.cc:348] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.628547 11065 raft_consensus.cc:374] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.628883 11065 raft_consensus.cc:729] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.629717 11065 consensus_queue.cc:260] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.631196 11067 raft_consensus.cc:348] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.631839 11067 raft_consensus.cc:374] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.632067 11067 raft_consensus.cc:729] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.632690 11067 consensus_queue.cc:260] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.633832 11072 tablet_bootstrap.cc:492] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:22.634438 11072 ts_tablet_manager.cc:1397] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.063s user 0.002s sys 0.019s
I20260809 15:51:22.637056 11065 ts_tablet_manager.cc:1428] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.011s user 0.003s sys 0.003s
I20260809 15:51:22.637863 11065 tablet_bootstrap.cc:492] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:22.638826 11067 ts_tablet_manager.cc:1428] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.010s user 0.006s sys 0.001s
I20260809 15:51:22.639714 11067 tablet_bootstrap.cc:492] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:22.640976 10737 catalog_manager.cc:5495] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:22.645494 11072 raft_consensus.cc:348] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.646214 11072 raft_consensus.cc:374] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.646557 11072 raft_consensus.cc:729] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.647404 11072 consensus_queue.cc:260] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.649407 11072 ts_tablet_manager.cc:1428] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.014s user 0.004s sys 0.000s
I20260809 15:51:22.650223 11072 tablet_bootstrap.cc:492] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:22.653808 11067 tablet_bootstrap.cc:654] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.655519 11065 tablet_bootstrap.cc:654] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.656273 11072 tablet_bootstrap.cc:654] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.671768 10929 raft_consensus.cc:3037] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:22.676368 11083 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:22.676991 11083 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.680932 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:22.682253 11083 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:51:22.683418 10893 leader_election.cc:341] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:22.683660 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:22.684360 11017 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:22.686542 10891 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:22.688323 11083 raft_consensus.cc:2781] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:22.688704 11083 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:22.689067 11083 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:22.691771 11067 tablet_bootstrap.cc:492] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:22.692304 11067 ts_tablet_manager.cc:1397] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.053s user 0.012s sys 0.002s
I20260809 15:51:22.695516 11067 raft_consensus.cc:348] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.696061 11072 tablet_bootstrap.cc:492] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:22.698727 11083 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.703379 11083 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:22.703817 11065 tablet_bootstrap.cc:492] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:22.701032 11067 raft_consensus.cc:374] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.701277 11072 ts_tablet_manager.cc:1397] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.051s user 0.007s sys 0.015s
I20260809 15:51:22.704262 11065 ts_tablet_manager.cc:1397] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.067s user 0.011s sys 0.006s
I20260809 15:51:22.704725 11067 raft_consensus.cc:729] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.704516 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:22.705368 11067 consensus_queue.cc:260] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:22.705896 10893 leader_election.cc:341] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:22.705891 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:22.711549 11017 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:22.712598 11072 raft_consensus.cc:348] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.713245 11072 raft_consensus.cc:374] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.713466 11072 raft_consensus.cc:729] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.714031 11072 consensus_queue.cc:260] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.716418 11072 ts_tablet_manager.cc:1428] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.012s user 0.005s sys 0.001s
I20260809 15:51:22.717130 11067 ts_tablet_manager.cc:1428] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.025s user 0.009s sys 0.001s
I20260809 15:51:22.718024 11067 tablet_bootstrap.cc:492] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:22.719748 11072 tablet_bootstrap.cc:492] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:22.726665 11065 raft_consensus.cc:348] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.729234 11065 raft_consensus.cc:374] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.729528 11065 raft_consensus.cc:729] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.730247 11065 consensus_queue.cc:260] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.732518 11065 ts_tablet_manager.cc:1428] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.028s user 0.011s sys 0.000s
I20260809 15:51:22.735240 11065 tablet_bootstrap.cc:492] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:22.738683 11072 tablet_bootstrap.cc:654] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.740867 11067 tablet_bootstrap.cc:654] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.734282 11017 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:22.745949 10891 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:22.747098 11083 raft_consensus.cc:2781] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:22.747658 11083 raft_consensus.cc:686] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:22.748593 11083 consensus_queue.cc:237] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.752245 11065 tablet_bootstrap.cc:654] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.780189 11195 consensus_queue.cc:1035] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:22.800777 11072 tablet_bootstrap.cc:492] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:22.801275 11072 ts_tablet_manager.cc:1397] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.082s user 0.028s sys 0.028s
I20260809 15:51:22.819460 11083 consensus_queue.cc:1035] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:22.822302 10733 catalog_manager.cc:5495] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:22.850679 11258 raft_consensus.cc:482] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:22.851421 11258 raft_consensus.cc:504] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.852047 11072 raft_consensus.cc:348] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.852748 11072 raft_consensus.cc:374] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.852947 11065 tablet_bootstrap.cc:492] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:22.853083 11072 raft_consensus.cc:729] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.856750 11065 ts_tablet_manager.cc:1397] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.122s user 0.013s sys 0.008s
I20260809 15:51:22.856952 11072 consensus_queue.cc:260] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.864464 11065 raft_consensus.cc:348] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.865353 11065 raft_consensus.cc:374] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.865691 11065 raft_consensus.cc:729] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.867230 11072 ts_tablet_manager.cc:1428] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.066s user 0.000s sys 0.006s
I20260809 15:51:22.868104 11072 tablet_bootstrap.cc:492] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:22.871218 11258 leader_election.cc:288] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:22.875450 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:22.876159 10929 raft_consensus.cc:2454] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:22.876807 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:22.877509 10848 raft_consensus.cc:2454] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:22.880849 11195 consensus_queue.cc:1035] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:22.881325 10929 raft_consensus.cc:3037] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:22.883080 10978 leader_election.cc:302] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:51:22.886704 10789 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 15:51:22.889895 11258 raft_consensus.cc:2781] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:22.901417 11258 raft_consensus.cc:482] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:22.901764 11258 raft_consensus.cc:3037] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:22.880059 11065 consensus_queue.cc:260] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.911420 11065 ts_tablet_manager.cc:1428] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.054s user 0.007s sys 0.001s
I20260809 15:51:22.912885 11073 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:22.913455 11073 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.914361 11072 tablet_bootstrap.cc:654] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:22.916081 11126 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:22.916918 11126 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.918815 11126 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:22.921211 11067 tablet_bootstrap.cc:492] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:22.921429 11073 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:22.921641 11067 ts_tablet_manager.cc:1397] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.204s user 0.016s sys 0.008s
I20260809 15:51:22.929116 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:22.928476 11073 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:22.931376 11073 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.927714 11067 raft_consensus.cc:348] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.933353 11067 raft_consensus.cc:374] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:22.933665 11067 raft_consensus.cc:729] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:22.935746 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:22.938992 11067 consensus_queue.cc:260] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.939275 11073 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:22.939561 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:22.950111 10927 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:22.952325 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:22.953133 10978 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:22.954188 10978 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:22.941233 11067 ts_tablet_manager.cc:1428] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.017s user 0.004s sys 0.000s
I20260809 15:51:22.943467 11065 tablet_bootstrap.cc:492] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:22.956697 11067 tablet_bootstrap.cc:492] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:22.957945 11308 raft_consensus.cc:2726] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:22.958801 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:22.969540 10843 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:22.966686 10844 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:22.972144 10893 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:22.970201 11239 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:22.972869 11308 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:22.959964 11073 raft_consensus.cc:2781] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:22.973552 11308 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.972890 11239 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.978029 11308 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:22.979002 11239 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:22.980887 11072 tablet_bootstrap.cc:492] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:22.981307 11072 ts_tablet_manager.cc:1397] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.113s user 0.013s sys 0.008s
I20260809 15:51:22.975263 11126 raft_consensus.cc:2781] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:22.981770 11126 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:22.987636 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:22.988906 11258 raft_consensus.cc:504] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:22.989995 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:22.982978 11073 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:22.992962 11073 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:22.982508 11126 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.003283 11072 raft_consensus.cc:348] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.006445 11072 raft_consensus.cc:374] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:23.006971 11072 raft_consensus.cc:729] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:23.007153 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:23.007570 11017 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.007532 11072 consensus_queue.cc:260] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.008064 10844 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:23.017529 11065 tablet_bootstrap.cc:654] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:23.005620 11258 leader_election.cc:288] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:23.019446 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:23.021515 10844 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.022496 11014 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
W20260809 15:51:23.021024 10809 leader_election.cc:341] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:23.024595 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:23.025177 10843 raft_consensus.cc:3037] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:22.994319 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:23.028860 10978 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:23.029692 11309 raft_consensus.cc:2781] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:23.030202 10929 raft_consensus.cc:2454] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:23.030917 11309 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:23.031579 11309 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.032158 10811 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:23.033457 11239 raft_consensus.cc:2781] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:23.033795 11239 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:23.034130 11239 raft_consensus.cc:3037] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.059278 11072 ts_tablet_manager.cc:1428] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.078s user 0.010s sys 0.002s
I20260809 15:51:23.060305 11072 tablet_bootstrap.cc:492] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:23.072486 11083 consensus_queue.cc:1035] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:23.072952 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:23.073698 10929 raft_consensus.cc:3037] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.078510 11231 consensus_queue.cc:1035] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:23.091624 10929 raft_consensus.cc:2454] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:23.091312 11073 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.092381 11126 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.094617 10927 raft_consensus.cc:2379] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 1: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:51:23.094853 11067 tablet_bootstrap.cc:654] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:23.098299 11126 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:23.102478 11188 consensus_queue.cc:1035] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:23.103528 10978 leader_election.cc:302] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:23.104126 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:23.104467 11308 raft_consensus.cc:2781] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:23.105319 11308 raft_consensus.cc:686] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:23.106487 11308 consensus_queue.cc:237] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.089529 11239 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.089588 10843 raft_consensus.cc:2454] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:23.113312 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:23.117724 10847 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.120215 11065 tablet_bootstrap.cc:492] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:23.120618 11065 ts_tablet_manager.cc:1397] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.177s user 0.015s sys 0.015s
I20260809 15:51:23.121284 11072 tablet_bootstrap.cc:654] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:23.125298 11065 raft_consensus.cc:348] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.125751 11065 raft_consensus.cc:374] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:23.125908 11065 raft_consensus.cc:729] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:23.126520 11065 consensus_queue.cc:260] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.128852 11065 ts_tablet_manager.cc:1428] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.008s user 0.000s sys 0.007s
I20260809 15:51:23.130901 11065 tablet_bootstrap.cc:492] T 81af4312dd044dfa85b9067b2fe4974c P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:23.122241 11073 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:23.137244 11239 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:23.150375 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:23.150924 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:23.151249 10929 raft_consensus.cc:3037] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.151648 10927 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.158385 11309 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.160315 11018 raft_consensus.cc:2379] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 1: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:51:23.166044 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:23.166699 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:23.169054 10931 raft_consensus.cc:2379] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 1: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:51:23.181818 10847 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:23.185865 11072 tablet_bootstrap.cc:492] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:23.165798 11309 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:23.186450 11072 ts_tablet_manager.cc:1397] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.126s user 0.012s sys 0.009s
I20260809 15:51:23.166841 10843 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.198372 10978 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:23.198770 11065 tablet_bootstrap.cc:654] T 81af4312dd044dfa85b9067b2fe4974c P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:23.203955 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:23.193125 10927 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:23.205600 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:23.204147 11308 raft_consensus.cc:2781] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:23.209781 11308 raft_consensus.cc:686] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:23.211319 11308 consensus_queue.cc:237] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.216924 11072 raft_consensus.cc:348] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.223536 10843 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:23.226307 11067 tablet_bootstrap.cc:492] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:23.226878 11067 ts_tablet_manager.cc:1397] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.270s user 0.030s sys 0.004s
I20260809 15:51:23.227195 10929 raft_consensus.cc:2454] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:23.228226 10811 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:23.228987 11239 raft_consensus.cc:2781] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:23.229494 11239 raft_consensus.cc:686] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:23.230453 11239 consensus_queue.cc:237] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.217826 11072 raft_consensus.cc:374] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:23.231583 11072 raft_consensus.cc:729] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
W20260809 15:51:23.233426 10809 leader_election.cc:341] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:23.234526 10978 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:23.229628 11067 raft_consensus.cc:348] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.235533 11073 raft_consensus.cc:2726] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 15:51:23.235654 11067 raft_consensus.cc:374] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:23.237347 11067 raft_consensus.cc:729] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:23.237982 11067 consensus_queue.cc:260] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.238272 10733 catalog_manager.cc:5495] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:23.242513 11072 consensus_queue.cc:260] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.244091 11067 ts_tablet_manager.cc:1428] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.017s user 0.007s sys 0.000s
I20260809 15:51:23.244750 11067 tablet_bootstrap.cc:492] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:23.251368 11072 ts_tablet_manager.cc:1428] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.061s user 0.005s sys 0.002s
I20260809 15:51:23.256359 11072 tablet_bootstrap.cc:492] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:23.260707 10893 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:23.263808 11083 raft_consensus.cc:2781] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:23.264358 11083 raft_consensus.cc:686] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:23.265200 11083 consensus_queue.cc:237] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.268504 11072 tablet_bootstrap.cc:654] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:23.272210 11067 tablet_bootstrap.cc:654] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:23.273981 10733 catalog_manager.cc:5495] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 0 to 1, leader changed from <none> to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:23.280637 11067 tablet_bootstrap.cc:492] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:23.281046 11067 ts_tablet_manager.cc:1397] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.036s user 0.010s sys 0.000s
I20260809 15:51:23.283543 11065 tablet_bootstrap.cc:492] T 81af4312dd044dfa85b9067b2fe4974c P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:23.283315 11067 raft_consensus.cc:348] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.283954 11065 ts_tablet_manager.cc:1397] T 81af4312dd044dfa85b9067b2fe4974c P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.153s user 0.013s sys 0.001s
I20260809 15:51:23.283985 11067 raft_consensus.cc:374] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:23.284391 11067 raft_consensus.cc:729] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:23.285010 11067 consensus_queue.cc:260] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.286665 11065 raft_consensus.cc:348] T 81af4312dd044dfa85b9067b2fe4974c P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.287225 11067 ts_tablet_manager.cc:1428] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 15:51:23.287333 11065 raft_consensus.cc:374] T 81af4312dd044dfa85b9067b2fe4974c P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:23.287653 11065 raft_consensus.cc:729] T 81af4312dd044dfa85b9067b2fe4974c P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:23.288128 11067 tablet_bootstrap.cc:492] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:23.288334 11065 consensus_queue.cc:260] T 81af4312dd044dfa85b9067b2fe4974c P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.291235 11065 ts_tablet_manager.cc:1428] T 81af4312dd044dfa85b9067b2fe4974c P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260809 15:51:23.292901 11065 tablet_bootstrap.cc:492] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:23.293217 11067 tablet_bootstrap.cc:654] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:23.293471 11072 tablet_bootstrap.cc:492] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:23.293871 11072 ts_tablet_manager.cc:1397] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.038s user 0.006s sys 0.005s
I20260809 15:51:23.296175 11072 raft_consensus.cc:348] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.296787 11072 raft_consensus.cc:374] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:23.298326 11072 raft_consensus.cc:729] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:23.298933 11072 consensus_queue.cc:260] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.301465 11072 ts_tablet_manager.cc:1428] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.007s user 0.003s sys 0.001s
I20260809 15:51:23.302610 10843 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.289453 10737 catalog_manager.cc:5495] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:23.310773 11072 tablet_bootstrap.cc:492] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:23.312021 11231 consensus_queue.cc:1035] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:23.324034 11067 tablet_bootstrap.cc:492] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:23.324456 11067 ts_tablet_manager.cc:1397] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.036s user 0.010s sys 0.004s
I20260809 15:51:23.326498 11065 tablet_bootstrap.cc:654] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:23.329054 11067 raft_consensus.cc:348] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.329780 11067 raft_consensus.cc:374] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:23.330045 11067 raft_consensus.cc:729] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:23.330669 11067 consensus_queue.cc:260] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.333235 11067 ts_tablet_manager.cc:1428] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260809 15:51:23.334144 11067 tablet_bootstrap.cc:492] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:23.334818 11073 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:23.335395 11073 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.339354 10733 catalog_manager.cc:5495] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:23.343675 11072 tablet_bootstrap.cc:654] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:23.360906 11073 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:23.373585 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:23.380450 11308 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:23.381059 11308 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.389350 11067 tablet_bootstrap.cc:654] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:23.389883 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:23.390251 11083 consensus_queue.cc:1035] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:23.389894 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:23.384228 11308 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:51:23.397315 10978 leader_election.cc:341] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
W20260809 15:51:23.398006 10978 leader_election.cc:341] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:23.390661 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:23.398483 10978 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:23.398839 10929 raft_consensus.cc:2379] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 1: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:51:23.404016 10978 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:23.404716 11309 raft_consensus.cc:2726] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:23.405726 11309 raft_consensus.cc:3037] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.423619 11065 tablet_bootstrap.cc:492] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:23.424165 11065 ts_tablet_manager.cc:1397] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.131s user 0.016s sys 0.004s
I20260809 15:51:23.426764 11065 raft_consensus.cc:348] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.427395 11065 raft_consensus.cc:374] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:23.427625 11065 raft_consensus.cc:729] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:23.428174 11065 consensus_queue.cc:260] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.430030 11065 ts_tablet_manager.cc:1428] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260809 15:51:23.430922 11073 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:23.431408 11067 tablet_bootstrap.cc:492] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:23.431391 11073 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.432044 11067 ts_tablet_manager.cc:1397] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.098s user 0.012s sys 0.010s
I20260809 15:51:23.432929 11072 tablet_bootstrap.cc:492] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:23.433332 11072 ts_tablet_manager.cc:1397] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.123s user 0.017s sys 0.004s
I20260809 15:51:23.433355 11073 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:23.434624 11065 tablet_bootstrap.cc:492] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:23.439242 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:23.439939 10848 raft_consensus.cc:2454] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:23.441591 10978 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:23.441974 11072 raft_consensus.cc:348] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.442685 11072 raft_consensus.cc:374] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:23.442592 11073 raft_consensus.cc:2781] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:23.443037 11072 raft_consensus.cc:729] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:23.443231 11073 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:23.443698 11073 raft_consensus.cc:3037] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.443652 11072 consensus_queue.cc:260] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.444177 11067 raft_consensus.cc:348] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.444854 11067 raft_consensus.cc:374] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:23.445116 11067 raft_consensus.cc:729] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:23.445355 11065 tablet_bootstrap.cc:654] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:23.446592 11072 ts_tablet_manager.cc:1428] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.013s user 0.007s sys 0.002s
I20260809 15:51:23.447435 11072 tablet_bootstrap.cc:492] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:23.445942 11067 consensus_queue.cc:260] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.450847 11067 ts_tablet_manager.cc:1428] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.019s user 0.007s sys 0.000s
I20260809 15:51:23.450734 11126 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:23.451673 11067 tablet_bootstrap.cc:492] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:23.451503 11126 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.458683 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:23.454363 11126 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:23.484534 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:23.485376 10927 raft_consensus.cc:2454] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:23.486541 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:23.487643 10848 raft_consensus.cc:2454] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:23.510818 11072 tablet_bootstrap.cc:654] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:23.520491 11236 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:23.521396 11236 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.524639 11236 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:23.527130 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:51:23.528556 10811 leader_election.cc:341] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:23.536141 11073 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.421891 11309 raft_consensus.cc:2726] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:23.541522 11073 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:23.547459 10893 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:23.546308 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:23.552171 10842 raft_consensus.cc:3037] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.547206 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:23.555827 11018 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:23.558270 11083 raft_consensus.cc:2781] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:23.561832 11083 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:23.562227 11083 raft_consensus.cc:3037] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.562147 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:23.540661 11308 consensus_queue.cc:1035] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.019s
I20260809 15:51:23.558698 11016 raft_consensus.cc:2379] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 1: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:51:23.570760 11065 tablet_bootstrap.cc:492] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:23.573797 11067 tablet_bootstrap.cc:654] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:23.578994 11065 ts_tablet_manager.cc:1397] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.145s user 0.022s sys 0.012s
I20260809 15:51:23.580209 10809 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:23.582129 11236 raft_consensus.cc:2781] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:23.584332 11236 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:23.587735 11236 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.584477 10842 raft_consensus.cc:2454] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:23.581004 11083 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.604358 10931 raft_consensus.cc:2379] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 1: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:51:23.605508 11072 tablet_bootstrap.cc:492] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:23.605964 11072 ts_tablet_manager.cc:1397] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.159s user 0.009s sys 0.012s
I20260809 15:51:23.608032 11083 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:23.609553 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:23.610602 11017 raft_consensus.cc:2379] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 1: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:51:23.610399 11072 raft_consensus.cc:348] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.611202 11072 raft_consensus.cc:374] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:23.611501 11072 raft_consensus.cc:729] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:23.616137 11065 raft_consensus.cc:348] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.619066 11065 raft_consensus.cc:374] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:23.619555 11065 raft_consensus.cc:729] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:23.619421 11072 consensus_queue.cc:260] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.620657 11065 consensus_queue.cc:260] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.622773 10978 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:23.624835 11258 raft_consensus.cc:2781] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:23.626436 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:23.626837 11258 raft_consensus.cc:686] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:23.625065 11308 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:23.628247 11065 ts_tablet_manager.cc:1428] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.049s user 0.003s sys 0.005s
I20260809 15:51:23.630554 11065 tablet_bootstrap.cc:492] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:23.627686 10844 raft_consensus.cc:2379] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 1: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:51:23.628067 11308 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.627715 11258 consensus_queue.cc:237] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.630841 11072 ts_tablet_manager.cc:1428] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.025s user 0.006s sys 0.001s
I20260809 15:51:23.640533 11072 tablet_bootstrap.cc:492] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:23.642607 10893 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:23.643625 11083 raft_consensus.cc:2726] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 15:51:23.647583 11308 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:23.651782 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:23.650164 11236 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:23.653667 10978 leader_election.cc:341] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:23.653584 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:51:23.656690 10978 leader_election.cc:341] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:23.657022 10978 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:23.658473 11341 consensus_queue.cc:1035] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260809 15:51:23.657919 11308 raft_consensus.cc:2726] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:23.660358 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:23.661226 10931 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.663699 11072 tablet_bootstrap.cc:654] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:23.667508 11308 consensus_queue.cc:1035] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:23.668562 11067 tablet_bootstrap.cc:492] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:23.670403 11067 ts_tablet_manager.cc:1397] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.219s user 0.009s sys 0.018s
I20260809 15:51:23.674856 11067 raft_consensus.cc:348] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.675647 11067 raft_consensus.cc:374] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:23.675966 11067 raft_consensus.cc:729] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:23.676736 11067 consensus_queue.cc:260] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.679620 10931 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:23.685930 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:23.686770 11017 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.687691 11067 ts_tablet_manager.cc:1428] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.017s user 0.006s sys 0.001s
I20260809 15:51:23.688153 11341 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:23.688824 11341 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.707839 10811 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:23.709225 11341 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:23.695016 11067 tablet_bootstrap.cc:492] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:23.710055 11109 raft_consensus.cc:2781] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:23.710700 11109 raft_consensus.cc:686] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:23.711712 11109 consensus_queue.cc:237] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.689635 10737 catalog_manager.cc:5495] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:23.706979 11065 tablet_bootstrap.cc:654] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:23.682560 11236 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:23.718487 11309 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:23.721685 11309 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.724836 11083 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:23.725613 11083 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.727227 11309 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:23.727818 11083 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:23.727166 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:23.728565 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:23.729986 10978 leader_election.cc:341] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:23.724713 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:23.730829 10893 leader_election.cc:341] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:23.730017 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:23.738735 11072 tablet_bootstrap.cc:492] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:23.739248 11072 ts_tablet_manager.cc:1397] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.099s user 0.010s sys 0.012s
I20260809 15:51:23.736330 11083 raft_consensus.cc:482] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:51:23.742800 10891 leader_election.cc:341] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:23.746245 10891 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:23.746495 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:23.736202 10843 raft_consensus.cc:2454] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:23.748270 11126 raft_consensus.cc:2726] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:23.744815 11083 raft_consensus.cc:504] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.748605 10978 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:23.750476 11083 leader_election.cc:288] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:51:23.751930 10811 leader_election.cc:341] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:23.753448 11072 raft_consensus.cc:348] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.757910 11072 raft_consensus.cc:374] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:23.758330 11072 raft_consensus.cc:729] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:23.752345 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:23.762790 11016 raft_consensus.cc:2454] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:23.763407 11348 raft_consensus.cc:2781] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:23.761817 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:23.763887 11348 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:23.763471 11072 consensus_queue.cc:260] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.764263 11348 raft_consensus.cc:3037] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.767859 11072 ts_tablet_manager.cc:1428] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.028s user 0.008s sys 0.001s
I20260809 15:51:23.768785 11072 tablet_bootstrap.cc:492] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:23.771113 10891 leader_election.cc:302] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:51:23.775527 10893 leader_election.cc:341] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:23.776261 11083 raft_consensus.cc:2781] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:23.776772 11083 raft_consensus.cc:482] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:23.778471 11083 raft_consensus.cc:3037] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.779455 11017 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:23.781976 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:23.789129 11083 raft_consensus.cc:504] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.791419 11067 tablet_bootstrap.cc:654] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:23.792538 11065 tablet_bootstrap.cc:492] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:23.795439 11065 ts_tablet_manager.cc:1397] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.165s user 0.011s sys 0.007s
I20260809 15:51:23.799279 10733 catalog_manager.cc:5495] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 0 to 1, leader changed from <none> to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:23.797432 11126 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:51:23.801901 10809 leader_election.cc:341] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:23.806442 10809 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:23.814241 11065 raft_consensus.cc:348] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.815160 11065 raft_consensus.cc:374] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:23.815464 11065 raft_consensus.cc:729] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:23.816890 11341 raft_consensus.cc:2726] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:23.802829 11126 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.818924 11065 consensus_queue.cc:260] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.794384 11258 consensus_queue.cc:1035] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260809 15:51:23.828403 11126 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:23.830542 11083 leader_election.cc:288] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:23.830265 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:23.832891 11065 ts_tablet_manager.cc:1428] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.037s user 0.003s sys 0.007s
I20260809 15:51:23.833848 11065 tablet_bootstrap.cc:492] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:23.838670 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:23.843554 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
W20260809 15:51:23.840598 10893 leader_election.cc:341] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:23.844224 11016 raft_consensus.cc:3037] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.847923 11308 consensus_queue.cc:1035] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:23.848214 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:23.852193 11017 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:23.854720 10891 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:23.855639 11195 raft_consensus.cc:2781] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:23.856283 11195 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:23.856812 11195 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.861764 11016 raft_consensus.cc:2454] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:23.848620 11072 tablet_bootstrap.cc:654] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:23.849125 11083 consensus_queue.cc:1035] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:23.872056 11341 consensus_queue.cc:1035] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260809 15:51:23.871148 10891 leader_election.cc:302] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
W20260809 15:51:23.878518 10893 leader_election.cc:341] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:23.884490 11231 raft_consensus.cc:2781] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:23.885097 11231 raft_consensus.cc:686] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:23.888943 11065 tablet_bootstrap.cc:654] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:23.877382 11348 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:23.887221 11349 log.cc:924] Time spent T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.060s user 0.000s sys 0.001s
I20260809 15:51:23.897240 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:23.886477 11231 consensus_queue.cc:237] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.898097 11348 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:23.901918 10842 raft_consensus.cc:3037] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.904904 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
W20260809 15:51:23.919579 10978 leader_election.cc:341] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:23.932830 11083 consensus_queue.cc:1035] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:23.942209 10842 raft_consensus.cc:2454] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:23.944196 10978 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:23.945488 11309 raft_consensus.cc:2781] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:23.946501 11309 raft_consensus.cc:686] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:23.947551 11309 consensus_queue.cc:237] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.962128 11195 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.967929 11308 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:23.968716 11308 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:23.974148 11308 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:23.974762 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:23.975772 10842 raft_consensus.cc:2454] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:23.978160 10978 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:23.980036 11258 raft_consensus.cc:2781] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:23.980513 11258 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:23.981026 11258 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:23.992920 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:24.000576 11067 tablet_bootstrap.cc:492] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:24.001143 11067 ts_tablet_manager.cc:1397] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.306s user 0.014s sys 0.014s
W20260809 15:51:24.008360 10978 leader_election.cc:341] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:24.012260 11072 tablet_bootstrap.cc:492] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:24.012722 11072 ts_tablet_manager.cc:1397] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.244s user 0.020s sys 0.024s
I20260809 15:51:24.023890 11195 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:24.025812 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:24.026546 11018 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:24.029333 11072 raft_consensus.cc:348] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.036804 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:24.058283 11072 raft_consensus.cc:374] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:24.058617 11072 raft_consensus.cc:729] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:24.059104 11067 raft_consensus.cc:348] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.059278 11072 consensus_queue.cc:260] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.059795 11067 raft_consensus.cc:374] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:24.060184 11067 raft_consensus.cc:729] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:24.060750 11067 consensus_queue.cc:260] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.062275 11067 ts_tablet_manager.cc:1428] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.061s user 0.004s sys 0.003s
I20260809 15:51:24.063097 11067 tablet_bootstrap.cc:492] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:24.056200 11065 tablet_bootstrap.cc:492] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:24.064591 11065 ts_tablet_manager.cc:1397] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.231s user 0.029s sys 0.016s
I20260809 15:51:24.043896 10737 catalog_manager.cc:5495] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
W20260809 15:51:24.068416 10893 leader_election.cc:341] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:24.064275 11258 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.096792 11018 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:24.100497 10891 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:24.103793 11126 raft_consensus.cc:2781] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:24.104311 11126 raft_consensus.cc:686] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:24.104943 11126 consensus_queue.cc:237] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.115162 11065 raft_consensus.cc:348] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.115684 11065 raft_consensus.cc:374] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:24.115949 11065 raft_consensus.cc:729] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:24.116403 11065 consensus_queue.cc:260] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.119685 11072 ts_tablet_manager.cc:1428] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.107s user 0.010s sys 0.047s
I20260809 15:51:24.120425 11072 tablet_bootstrap.cc:492] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:24.110345 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:24.123461 10733 catalog_manager.cc:5495] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:24.141727 11067 tablet_bootstrap.cc:654] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:24.142159 10843 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:24.146950 11239 consensus_queue.cc:1035] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:24.149153 11258 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:24.154698 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
W20260809 15:51:24.156287 10978 leader_election.cc:341] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:24.156725 11072 tablet_bootstrap.cc:654] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:24.172500 11065 ts_tablet_manager.cc:1428] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.108s user 0.006s sys 0.002s
I20260809 15:51:24.174299 11309 consensus_queue.cc:1035] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:24.176208 11065 tablet_bootstrap.cc:492] T 974451500a6e434f8d407d9f178ab710 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:24.181293 11366 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:24.191983 11072 tablet_bootstrap.cc:492] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:24.192442 11072 ts_tablet_manager.cc:1397] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.072s user 0.011s sys 0.011s
I20260809 15:51:24.183248 11366 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.195672 11067 tablet_bootstrap.cc:492] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:24.196038 11067 ts_tablet_manager.cc:1397] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.133s user 0.017s sys 0.009s
I20260809 15:51:24.195335 11366 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:24.201920 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:24.201930 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:24.203683 10891 leader_election.cc:341] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
W20260809 15:51:24.204649 10893 leader_election.cc:341] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:24.205029 10893 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:24.205492 11067 raft_consensus.cc:348] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.208150 11067 raft_consensus.cc:374] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:24.208415 11067 raft_consensus.cc:729] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:24.209028 11067 consensus_queue.cc:260] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.209721 11366 raft_consensus.cc:2726] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:24.211030 11067 ts_tablet_manager.cc:1428] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.015s user 0.004s sys 0.003s
I20260809 15:51:24.211051 11072 raft_consensus.cc:348] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.211843 11067 tablet_bootstrap.cc:492] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:24.211894 11072 raft_consensus.cc:374] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:24.212217 11072 raft_consensus.cc:729] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:24.212832 10843 raft_consensus.cc:2454] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:24.215811 10978 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:24.213236 11072 consensus_queue.cc:260] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.217525 11309 raft_consensus.cc:2781] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:24.219605 11072 ts_tablet_manager.cc:1428] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.027s user 0.002s sys 0.006s
I20260809 15:51:24.219707 11065 tablet_bootstrap.cc:654] T 974451500a6e434f8d407d9f178ab710 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:24.219481 11309 raft_consensus.cc:686] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:24.222754 11309 consensus_queue.cc:237] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.226769 11072 tablet_bootstrap.cc:492] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:24.225960 10737 catalog_manager.cc:5495] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:24.228716 11346 consensus_queue.cc:1035] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260809 15:51:24.239895 11341 consensus_queue.cc:1035] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:24.284343 11346 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:24.284107 11366 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:24.285127 11366 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.285009 11346 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.300266 11067 tablet_bootstrap.cc:654] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:24.300571 11346 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:24.302580 11072 tablet_bootstrap.cc:654] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:24.303120 11366 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:24.306974 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:24.307587 11017 raft_consensus.cc:2454] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:24.308337 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:24.304360 11231 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:51:24.310328 10893 leader_election.cc:341] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:24.309933 11231 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.312660 11231 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:24.314040 10891 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:24.315371 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:24.316144 11018 raft_consensus.cc:2454] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:24.316546 11366 consensus_queue.cc:1035] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:24.319407 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:24.319612 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:24.320030 10843 raft_consensus.cc:2454] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:24.320168 10929 raft_consensus.cc:2454] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:24.323246 10978 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:24.323930 10893 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:24.316169 11231 raft_consensus.cc:2781] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:24.331393 10737 catalog_manager.cc:5495] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:24.326766 11309 raft_consensus.cc:2781] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:24.326961 11366 raft_consensus.cc:2781] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:24.328262 11231 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:24.335669 11309 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:24.337049 11231 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:24.337260 11309 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:24.336493 11366 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:24.345468 11366 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:24.333693 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:24.351074 10843 raft_consensus.cc:2454] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:24.354836 11341 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:24.355368 11341 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.357743 11341 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:24.358295 11309 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.361219 11309 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:24.364683 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:24.365483 10929 raft_consensus.cc:2454] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:24.387555 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:24.389518 10811 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:24.389882 11065 tablet_bootstrap.cc:492] T 974451500a6e434f8d407d9f178ab710 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:24.390378 11065 ts_tablet_manager.cc:1397] T 974451500a6e434f8d407d9f178ab710 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.217s user 0.022s sys 0.018s
W20260809 15:51:24.397028 10978 consensus_peers.cc:487] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa -> Peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Couldn't send request to peer 20730d75ad954a82b126148f8152a9f3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:24.407003 11065 raft_consensus.cc:348] T 974451500a6e434f8d407d9f178ab710 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.407694 11065 raft_consensus.cc:374] T 974451500a6e434f8d407d9f178ab710 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:24.407953 11065 raft_consensus.cc:729] T 974451500a6e434f8d407d9f178ab710 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:24.408063 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:24.408635 10842 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:24.408542 11065 consensus_queue.cc:260] T 974451500a6e434f8d407d9f178ab710 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.409593 11340 raft_consensus.cc:2781] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:24.409979 11340 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:24.410411 11340 raft_consensus.cc:3037] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:24.414562 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:24.432885 11366 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.461324 10929 raft_consensus.cc:2379] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 1: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:51:24.470229 11309 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:24.470832 11309 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.478070 11340 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.401043 11231 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.499076 11067 tablet_bootstrap.cc:492] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:24.499655 11067 ts_tablet_manager.cc:1397] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.288s user 0.011s sys 0.013s
W20260809 15:51:24.502499 10789 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 15:51:24.504211 11067 raft_consensus.cc:348] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.507131 11067 raft_consensus.cc:374] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:24.507512 11067 raft_consensus.cc:729] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:24.508339 11067 consensus_queue.cc:260] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.510701 11067 ts_tablet_manager.cc:1428] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s
I20260809 15:51:24.511931 11067 tablet_bootstrap.cc:492] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:24.492215 11072 tablet_bootstrap.cc:492] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:24.513880 11072 ts_tablet_manager.cc:1397] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.293s user 0.016s sys 0.003s
W20260809 15:51:24.493103 10809 leader_election.cc:341] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:24.487766 11065 ts_tablet_manager.cc:1428] T 974451500a6e434f8d407d9f178ab710 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.097s user 0.004s sys 0.024s
I20260809 15:51:24.515239 11065 tablet_bootstrap.cc:492] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:24.485759 10842 raft_consensus.cc:2454] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:24.491546 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:24.491457 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 102066us
I20260809 15:51:24.519287 10978 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:51:24.520304 10978 leader_election.cc:341] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:24.494284 11309 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:24.527411 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:24.528146 11072 raft_consensus.cc:348] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.530503 11072 raft_consensus.cc:374] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:24.523440 11348 raft_consensus.cc:2781] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:24.530814 11072 raft_consensus.cc:729] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:24.531111 11348 raft_consensus.cc:686] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:24.531456 11072 consensus_queue.cc:260] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.532320 11348 consensus_queue.cc:237] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.528290 11018 raft_consensus.cc:3037] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:24.523516 11340 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:24.561358 11258 consensus_queue.cc:1035] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:24.575768 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:24.576673 10848 raft_consensus.cc:2379] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 1: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:51:24.578608 10843 raft_consensus.cc:3037] T 974451500a6e434f8d407d9f178ab710 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:24.578783 11231 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:24.586606 11072 ts_tablet_manager.cc:1428] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.072s user 0.009s sys 0.031s
I20260809 15:51:24.587427 11065 tablet_bootstrap.cc:654] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:24.588171 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:24.589773 11072 tablet_bootstrap.cc:492] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
W20260809 15:51:24.590200 10978 leader_election.cc:341] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:24.590696 10978 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:24.592044 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:51:24.604764 10893 leader_election.cc:341] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:24.613693 11067 tablet_bootstrap.cc:654] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:24.601331 11258 raft_consensus.cc:2726] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:24.622931 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:24.623720 10929 raft_consensus.cc:3037] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:24.631491 11366 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:24.639673 11018 raft_consensus.cc:2454] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:24.640913 10809 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:24.645449 11384 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:24.645325 10929 raft_consensus.cc:2454] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:24.644347 11340 raft_consensus.cc:2781] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:24.646204 11384 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.647300 11340 raft_consensus.cc:686] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:24.654840 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:24.656045 10893 leader_election.cc:341] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:24.662386 11384 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:24.665055 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:24.665585 11016 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:24.648656 11340 consensus_queue.cc:237] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.678151 10736 catalog_manager.cc:5495] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:24.683692 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:24.696857 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:24.701123 11382 consensus_queue.cc:1035] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260809 15:51:24.707295 10891 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:24.708714 11382 raft_consensus.cc:2726] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 15:51:24.698437 11383 consensus_queue.cc:1035] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.015s
W20260809 15:51:24.728603 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 65810.1us
I20260809 15:51:24.730784 11072 tablet_bootstrap.cc:654] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
W20260809 15:51:24.740221 10893 consensus_peers.cc:487] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 -> Peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Couldn't send request to peer a53441872c0c416b8e2f26f52399f536. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260809 15:51:24.740186 10891 leader_election.cc:341] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:24.740924 10891 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:24.746564 11384 raft_consensus.cc:2726] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:24.748675 11016 raft_consensus.cc:2454] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:24.752504 11065 tablet_bootstrap.cc:492] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:24.753232 10891 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
W20260809 15:51:24.754936 10978 consensus_peers.cc:487] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa -> Peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Couldn't send request to peer 20730d75ad954a82b126148f8152a9f3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:24.755012 11383 raft_consensus.cc:2781] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:24.755762 11383 raft_consensus.cc:686] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:24.753484 11065 ts_tablet_manager.cc:1397] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.238s user 0.010s sys 0.026s
I20260809 15:51:24.754922 11067 tablet_bootstrap.cc:492] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:24.756858 11383 consensus_queue.cc:237] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.763613 11067 ts_tablet_manager.cc:1397] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.252s user 0.012s sys 0.012s
I20260809 15:51:24.781013 11065 raft_consensus.cc:348] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.781780 11065 raft_consensus.cc:374] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:24.782063 11065 raft_consensus.cc:729] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:24.786098 11065 consensus_queue.cc:260] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.787081 11067 raft_consensus.cc:348] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.787810 11067 raft_consensus.cc:374] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:24.788089 11067 raft_consensus.cc:729] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:24.787070 10733 catalog_manager.cc:5495] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:24.794164 11067 consensus_queue.cc:260] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.796002 11065 ts_tablet_manager.cc:1428] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.038s user 0.005s sys 0.002s
I20260809 15:51:24.796741 11065 tablet_bootstrap.cc:492] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:24.800911 11067 ts_tablet_manager.cc:1428] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.037s user 0.006s sys 0.000s
I20260809 15:51:24.801307 11073 consensus_queue.cc:1035] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:24.801697 11067 tablet_bootstrap.cc:492] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:24.819722 10737 catalog_manager.cc:5495] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 0 to 1, leader changed from <none> to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:24.826711 11072 tablet_bootstrap.cc:492] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:24.827214 11072 ts_tablet_manager.cc:1397] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.238s user 0.014s sys 0.005s
I20260809 15:51:24.831295 11072 raft_consensus.cc:348] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.832170 11072 raft_consensus.cc:374] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:24.832500 11072 raft_consensus.cc:729] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:24.835661 11072 consensus_queue.cc:260] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.838250 11072 ts_tablet_manager.cc:1428] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.011s user 0.007s sys 0.001s
I20260809 15:51:24.842840 11072 tablet_bootstrap.cc:492] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:24.873426 11065 tablet_bootstrap.cc:654] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:24.880924 11067 tablet_bootstrap.cc:654] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:24.883424 11072 tablet_bootstrap.cc:654] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:24.905921 11067 tablet_bootstrap.cc:492] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:24.907449 11067 ts_tablet_manager.cc:1397] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.106s user 0.010s sys 0.003s
I20260809 15:51:24.909984 11067 raft_consensus.cc:348] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.914722 11067 raft_consensus.cc:374] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:24.915045 11067 raft_consensus.cc:729] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:24.915759 11067 consensus_queue.cc:260] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.918843 11065 tablet_bootstrap.cc:492] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:24.919296 11065 ts_tablet_manager.cc:1397] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.123s user 0.007s sys 0.010s
I20260809 15:51:24.919605 11067 ts_tablet_manager.cc:1428] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260809 15:51:24.921527 11065 raft_consensus.cc:348] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.922029 11067 tablet_bootstrap.cc:492] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:24.922264 11065 raft_consensus.cc:374] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:24.922567 11065 raft_consensus.cc:729] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:24.923225 11065 consensus_queue.cc:260] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.928820 11067 tablet_bootstrap.cc:654] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:24.933643 11072 tablet_bootstrap.cc:492] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:24.933840 11065 ts_tablet_manager.cc:1428] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260809 15:51:24.934191 11072 ts_tablet_manager.cc:1397] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.092s user 0.009s sys 0.013s
I20260809 15:51:24.937255 11072 raft_consensus.cc:348] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.937950 11072 raft_consensus.cc:374] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:24.938253 11072 raft_consensus.cc:729] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:24.938512 11065 tablet_bootstrap.cc:492] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:24.938979 11072 consensus_queue.cc:260] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.941586 11072 ts_tablet_manager.cc:1428] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260809 15:51:24.943655 11072 tablet_bootstrap.cc:492] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:24.956707 11065 tablet_bootstrap.cc:654] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:24.957193 11072 tablet_bootstrap.cc:654] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:24.968012 11067 tablet_bootstrap.cc:492] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:24.968598 11067 ts_tablet_manager.cc:1397] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.047s user 0.007s sys 0.015s
I20260809 15:51:24.980854 11341 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:24.981495 11341 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:24.984727 11341 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:24.995091 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:24.996953 11258 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:51:24.997928 10811 leader_election.cc:341] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:24.997649 11258 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.000171 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:25.000890 11016 raft_consensus.cc:2454] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:25.002583 10809 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:25.003557 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:25.003787 11258 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:25.004820 11388 raft_consensus.cc:2781] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:25.005497 11388 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:25.006035 11388 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:25.013572 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:51:25.017242 10978 leader_election.cc:341] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
W20260809 15:51:25.021571 10978 leader_election.cc:341] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:25.022130 10978 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:25.023113 11258 raft_consensus.cc:2726] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:25.024897 11383 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:25.025722 11383 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.028550 11383 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:25.030690 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:51:25.032747 10891 leader_election.cc:341] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:25.039379 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:25.041885 10893 leader_election.cc:341] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:25.042457 10893 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:25.044682 11382 raft_consensus.cc:2726] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:25.067488 11309 consensus_queue.cc:1035] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:25.082360 11258 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:25.083191 11258 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.086022 11258 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:25.090358 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:25.093976 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:51:25.115182 10978 leader_election.cc:341] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
W20260809 15:51:25.125226 10978 leader_election.cc:341] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:25.125746 10978 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:25.112967 11067 raft_consensus.cc:348] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.127065 11067 raft_consensus.cc:374] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:25.127432 11067 raft_consensus.cc:729] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:25.133615 11348 raft_consensus.cc:2726] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:25.128286 11067 consensus_queue.cc:260] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.162588 11346 consensus_queue.cc:1035] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260809 15:51:25.168844 11340 consensus_queue.cc:1035] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:25.175752 11340 consensus_queue.cc:1035] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:25.185289 10848 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:25.187976 11388 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.193552 11072 tablet_bootstrap.cc:492] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:25.194128 11072 ts_tablet_manager.cc:1397] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.251s user 0.056s sys 0.116s
I20260809 15:51:25.193789 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:25.199899 11388 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:25.201326 11067 ts_tablet_manager.cc:1428] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.232s user 0.006s sys 0.006s
I20260809 15:51:25.201510 11072 raft_consensus.cc:348] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.202337 11072 raft_consensus.cc:374] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:25.202663 11072 raft_consensus.cc:729] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:25.203722 11067 tablet_bootstrap.cc:492] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:25.203276 11072 consensus_queue.cc:260] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.208812 11072 ts_tablet_manager.cc:1428] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.014s user 0.005s sys 0.001s
I20260809 15:51:25.209628 11072 tablet_bootstrap.cc:492] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:25.210891 11384 consensus_queue.cc:1035] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:25.211459 10926 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:25.202978 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:25.216620 11014 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:25.218961 11065 tablet_bootstrap.cc:492] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:25.219434 11065 ts_tablet_manager.cc:1397] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.281s user 0.061s sys 0.078s
W20260809 15:51:25.222581 10893 consensus_peers.cc:487] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 -> Peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Couldn't send request to peer a53441872c0c416b8e2f26f52399f536. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:25.224555 11072 tablet_bootstrap.cc:654] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:25.234470 11072 tablet_bootstrap.cc:492] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:25.236186 11072 ts_tablet_manager.cc:1397] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.027s user 0.010s sys 0.000s
I20260809 15:51:25.238924 11072 raft_consensus.cc:348] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.239724 11072 raft_consensus.cc:374] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:25.239988 11072 raft_consensus.cc:729] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:25.221815 11065 raft_consensus.cc:348] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.241887 11065 raft_consensus.cc:374] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:25.242180 11065 raft_consensus.cc:729] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:25.247440 11065 consensus_queue.cc:260] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.242589 11072 consensus_queue.cc:260] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.266988 11072 ts_tablet_manager.cc:1428] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.031s user 0.007s sys 0.000s
I20260809 15:51:25.268338 10926 raft_consensus.cc:2454] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:25.271570 11072 tablet_bootstrap.cc:492] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:25.273000 10811 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:25.263517 11384 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:25.275703 11384 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.276335 11341 raft_consensus.cc:2781] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:25.277148 11341 raft_consensus.cc:686] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:25.278316 11341 consensus_queue.cc:237] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.279672 11384 leader_election.cc:288] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:25.284320 10931 raft_consensus.cc:3037] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:25.289566 11014 raft_consensus.cc:2454] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:25.293957 11067 tablet_bootstrap.cc:654] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:25.285017 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:51:25.301138 10891 leader_election.cc:341] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:25.314561 11072 tablet_bootstrap.cc:654] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:25.328269 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:25.336107 10893 leader_election.cc:341] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:25.336591 10893 leader_election.cc:302] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:25.337702 11384 raft_consensus.cc:2726] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:25.341864 10736 catalog_manager.cc:5495] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 0 to 1, leader changed from <none> to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:25.347280 11065 ts_tablet_manager.cc:1428] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.128s user 0.006s sys 0.000s
I20260809 15:51:25.348251 11065 tablet_bootstrap.cc:492] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:25.384723 11065 tablet_bootstrap.cc:654] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:25.385566 11067 tablet_bootstrap.cc:492] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:25.386051 11067 ts_tablet_manager.cc:1397] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.183s user 0.016s sys 0.009s
I20260809 15:51:25.389314 11067 raft_consensus.cc:348] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.390151 11067 raft_consensus.cc:374] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:25.390599 11067 raft_consensus.cc:729] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:25.395702 11067 consensus_queue.cc:260] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.400339 11072 tablet_bootstrap.cc:492] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:25.400776 11072 ts_tablet_manager.cc:1397] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.129s user 0.019s sys 0.007s
I20260809 15:51:25.403179 11072 raft_consensus.cc:348] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.404138 11067 ts_tablet_manager.cc:1428] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.018s user 0.005s sys 0.003s
I20260809 15:51:25.404939 11067 tablet_bootstrap.cc:492] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:25.415664 11065 tablet_bootstrap.cc:492] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:25.416103 11065 ts_tablet_manager.cc:1397] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.068s user 0.009s sys 0.020s
I20260809 15:51:25.422010 11388 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:25.422524 11388 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.424427 11388 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:25.425612 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:25.426812 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:25.427423 10931 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
W20260809 15:51:25.428030 10809 leader_election.cc:341] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:25.403883 11072 raft_consensus.cc:374] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:25.428815 10811 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:25.429070 11072 raft_consensus.cc:729] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:25.429996 11072 consensus_queue.cc:260] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.430378 11340 raft_consensus.cc:2781] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:25.431623 11340 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:25.431945 11340 raft_consensus.cc:3037] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:25.432132 11072 ts_tablet_manager.cc:1428] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.031s user 0.005s sys 0.000s
I20260809 15:51:25.434603 11072 tablet_bootstrap.cc:492] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:25.438334 11067 tablet_bootstrap.cc:654] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:25.445273 11072 tablet_bootstrap.cc:654] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:25.450712 11340 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.453456 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:25.454242 10931 raft_consensus.cc:3037] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:25.454457 11065 raft_consensus.cc:348] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.455272 11065 raft_consensus.cc:374] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:25.455559 11065 raft_consensus.cc:729] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:25.456182 11065 consensus_queue.cc:260] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.460011 11340 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:25.461946 11065 ts_tablet_manager.cc:1428] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.046s user 0.003s sys 0.003s
I20260809 15:51:25.462859 11065 tablet_bootstrap.cc:492] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:25.463306 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
W20260809 15:51:25.465835 10809 leader_election.cc:341] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:25.472060 11065 tablet_bootstrap.cc:654] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:25.477782 10931 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:25.480943 11366 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:25.481093 11072 tablet_bootstrap.cc:492] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:25.481627 11072 ts_tablet_manager.cc:1397] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.047s user 0.011s sys 0.006s
I20260809 15:51:25.481716 10811 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:25.481468 11366 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.483351 11067 tablet_bootstrap.cc:492] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:25.483765 11067 ts_tablet_manager.cc:1397] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.079s user 0.017s sys 0.024s
I20260809 15:51:25.483966 11366 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:25.485610 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:25.486236 11016 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:25.486119 11067 raft_consensus.cc:348] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.486775 11067 raft_consensus.cc:374] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:25.487057 11067 raft_consensus.cc:729] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:25.487757 10891 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:25.487957 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:25.488660 11340 raft_consensus.cc:2781] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:25.489114 11340 raft_consensus.cc:686] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
W20260809 15:51:25.489614 10893 leader_election.cc:341] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:25.489518 11072 raft_consensus.cc:348] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.490231 11072 raft_consensus.cc:374] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:25.489874 11340 consensus_queue.cc:237] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.490494 11072 raft_consensus.cc:729] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:25.491111 11072 consensus_queue.cc:260] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.492107 11384 raft_consensus.cc:2781] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:25.492575 11384 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:25.492950 11384 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:25.493233 11072 ts_tablet_manager.cc:1428] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.011s user 0.003s sys 0.000s
I20260809 15:51:25.493850 11067 consensus_queue.cc:260] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.496503 11067 ts_tablet_manager.cc:1428] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.012s user 0.006s sys 0.000s
I20260809 15:51:25.497349 11072 tablet_bootstrap.cc:492] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:25.498693 11067 tablet_bootstrap.cc:492] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:25.504253 11067 tablet_bootstrap.cc:654] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:25.508111 11384 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.513993 11072 tablet_bootstrap.cc:654] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:25.517861 10736 catalog_manager.cc:5495] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 0 to 1, leader changed from <none> to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:25.519122 11067 tablet_bootstrap.cc:492] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:25.519538 11067 ts_tablet_manager.cc:1397] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.021s user 0.008s sys 0.000s
I20260809 15:51:25.520848 11346 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:25.521275 11346 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.521752 11067 raft_consensus.cc:348] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.522488 11067 raft_consensus.cc:374] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:25.522732 11067 raft_consensus.cc:729] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:25.523070 11346 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:25.523293 11067 consensus_queue.cc:260] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.525292 11067 ts_tablet_manager.cc:1428] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.005s user 0.002s sys 0.002s
I20260809 15:51:25.525388 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:25.526152 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:51:25.527472 10978 leader_election.cc:341] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
W20260809 15:51:25.528115 10978 leader_election.cc:341] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:25.528462 10978 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:25.529083 11072 tablet_bootstrap.cc:492] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:25.529170 11309 raft_consensus.cc:2726] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:25.529505 11072 ts_tablet_manager.cc:1397] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.032s user 0.009s sys 0.000s
I20260809 15:51:25.531814 11072 raft_consensus.cc:348] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.532475 11072 raft_consensus.cc:374] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:25.532761 11072 raft_consensus.cc:729] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:25.533319 11072 consensus_queue.cc:260] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.535251 11067 tablet_bootstrap.cc:492] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:25.537360 11072 ts_tablet_manager.cc:1428] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260809 15:51:25.538266 11072 tablet_bootstrap.cc:492] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:25.543488 11072 tablet_bootstrap.cc:654] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:25.544225 11065 tablet_bootstrap.cc:492] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:25.544643 11065 ts_tablet_manager.cc:1397] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.082s user 0.007s sys 0.013s
I20260809 15:51:25.546833 11065 raft_consensus.cc:348] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.547459 11065 raft_consensus.cc:374] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:25.547696 11065 raft_consensus.cc:729] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:25.549721 11067 tablet_bootstrap.cc:654] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:25.550146 11065 consensus_queue.cc:260] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.552223 11065 ts_tablet_manager.cc:1428] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260809 15:51:25.557173 11065 tablet_bootstrap.cc:492] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:25.562294 11065 tablet_bootstrap.cc:654] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:25.571244 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:25.565089 11072 tablet_bootstrap.cc:492] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:25.572386 11072 ts_tablet_manager.cc:1397] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.034s user 0.009s sys 0.005s
I20260809 15:51:25.574978 11072 raft_consensus.cc:348] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.575627 11072 raft_consensus.cc:374] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:25.575892 11072 raft_consensus.cc:729] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:25.576463 11072 consensus_queue.cc:260] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.567121 11067 tablet_bootstrap.cc:492] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:25.578347 11067 ts_tablet_manager.cc:1397] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.052s user 0.012s sys 0.003s
I20260809 15:51:25.567430 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:25.580665 11067 raft_consensus.cc:348] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.571828 11016 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:25.581367 11067 raft_consensus.cc:374] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:25.581627 11067 raft_consensus.cc:729] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:25.582305 11067 consensus_queue.cc:260] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.568992 11384 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:25.584268 11067 ts_tablet_manager.cc:1428] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.006s user 0.003s sys 0.001s
I20260809 15:51:25.585068 11067 tablet_bootstrap.cc:492] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
W20260809 15:51:25.581018 10893 leader_election.cc:341] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:25.586614 11072 ts_tablet_manager.cc:1428] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.014s user 0.006s sys 0.000s
I20260809 15:51:25.587472 11072 tablet_bootstrap.cc:492] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:25.588858 11016 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:25.589875 10891 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:25.590728 11384 raft_consensus.cc:2781] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:25.591404 11384 raft_consensus.cc:686] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:25.596704 11067 tablet_bootstrap.cc:654] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:25.596172 11384 consensus_queue.cc:237] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:25.598989 10789 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 15:51:25.602638 11065 tablet_bootstrap.cc:492] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:25.603137 11065 ts_tablet_manager.cc:1397] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.046s user 0.008s sys 0.002s
I20260809 15:51:25.592573 11072 tablet_bootstrap.cc:654] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:25.607096 11065 raft_consensus.cc:348] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.608183 11065 raft_consensus.cc:374] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:25.608453 11065 raft_consensus.cc:729] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:25.609257 11065 consensus_queue.cc:260] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.616154 11065 ts_tablet_manager.cc:1428] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.013s user 0.008s sys 0.000s
I20260809 15:51:25.617256 11065 tablet_bootstrap.cc:492] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:25.641732 11065 tablet_bootstrap.cc:654] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:25.663126 11258 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:25.663767 11258 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.666965 11346 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:25.667641 11346 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.672260 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:25.676051 10978 leader_election.cc:341] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:25.690603 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:25.694906 11366 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:25.695556 11366 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:25.698144 10978 leader_election.cc:341] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:25.701428 11258 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:25.702684 11346 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:25.707567 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:25.712440 11366 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:51:25.731406 10978 leader_election.cc:341] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:25.731869 10978 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:25.737051 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:25.732920 11258 raft_consensus.cc:2726] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:25.748126 11067 tablet_bootstrap.cc:492] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:25.748641 11067 ts_tablet_manager.cc:1397] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.164s user 0.039s sys 0.052s
I20260809 15:51:25.741647 10929 raft_consensus.cc:2454] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:25.741662 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:25.751091 11067 raft_consensus.cc:348] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.751492 10844 raft_consensus.cc:2454] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:25.751749 11067 raft_consensus.cc:374] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:25.752036 11067 raft_consensus.cc:729] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:25.752619 11067 consensus_queue.cc:260] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.755916 10893 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:25.757083 10978 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:25.759027 11067 ts_tablet_manager.cc:1428] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.010s user 0.006s sys 0.001s
I20260809 15:51:25.759945 11067 tablet_bootstrap.cc:492] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:25.764451 11258 raft_consensus.cc:2781] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:25.761969 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:25.769910 11340 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:25.770780 11258 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:25.775399 11258 raft_consensus.cc:3037] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:25.775229 11340 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.776114 11065 tablet_bootstrap.cc:492] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:25.767884 11388 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.009s
I20260809 15:51:25.776562 11065 ts_tablet_manager.cc:1397] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.160s user 0.022s sys 0.005s
I20260809 15:51:25.777182 11340 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:25.780292 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:25.781292 11065 raft_consensus.cc:348] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.782362 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:25.782068 11065 raft_consensus.cc:374] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:25.783468 11065 raft_consensus.cc:729] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:25.786530 11072 tablet_bootstrap.cc:492] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:25.769138 11442 raft_consensus.cc:2781] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:25.786970 11072 ts_tablet_manager.cc:1397] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.200s user 0.017s sys 0.008s
I20260809 15:51:25.791438 10736 catalog_manager.cc:5495] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:25.777702 11016 raft_consensus.cc:2454] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:25.785751 11065 consensus_queue.cc:260] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.787665 10811 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:25.802496 11239 raft_consensus.cc:2726] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:25.803231 11065 ts_tablet_manager.cc:1428] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.026s user 0.007s sys 0.000s
I20260809 15:51:25.804010 11065 tablet_bootstrap.cc:492] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:25.792042 11442 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:25.808092 11442 raft_consensus.cc:3037] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:25.787401 11382 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:25.809541 11072 raft_consensus.cc:348] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.797952 11067 tablet_bootstrap.cc:654] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:25.812266 11072 raft_consensus.cc:374] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:25.812726 11072 raft_consensus.cc:729] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:25.818181 11072 consensus_queue.cc:260] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.820402 11065 tablet_bootstrap.cc:654] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:25.821733 11340 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:25.812945 11382 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.834822 11072 ts_tablet_manager.cc:1428] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.044s user 0.008s sys 0.000s
I20260809 15:51:25.836424 10929 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:25.841378 11072 tablet_bootstrap.cc:492] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:25.824620 11340 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.857154 10841 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:25.857828 11258 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.877535 11258 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:25.874830 11442 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.878763 11340 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:25.875998 11382 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:25.881428 11072 tablet_bootstrap.cc:654] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:25.882694 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:25.880152 11442 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:25.887570 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:25.899466 11067 tablet_bootstrap.cc:492] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:25.900028 11067 ts_tablet_manager.cc:1397] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.140s user 0.007s sys 0.013s
I20260809 15:51:25.902920 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:25.903113 11388 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:25.886777 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:25.906698 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:25.907425 10848 raft_consensus.cc:3037] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:25.886931 10931 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:25.901661 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:25.916005 10848 raft_consensus.cc:2454] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:25.904603 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:25.911593 11018 raft_consensus.cc:3037] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:51:25.917516 10978 leader_election.cc:341] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:25.919938 10811 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:25.921657 10930 raft_consensus.cc:3037] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:25.928210 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:25.928804 11011 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:25.928782 11067 raft_consensus.cc:348] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.929558 11067 raft_consensus.cc:374] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:25.929812 11067 raft_consensus.cc:729] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:25.930642 11067 consensus_queue.cc:260] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.932955 11067 ts_tablet_manager.cc:1428] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.033s user 0.006s sys 0.001s
I20260809 15:51:25.938683 11340 raft_consensus.cc:2781] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:25.939523 11340 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:25.939843 11340 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:25.910874 11388 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.956952 11067 tablet_bootstrap.cc:492] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:25.957067 11444 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:25.957939 11444 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:25.968449 11444 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:25.969394 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:25.970062 11016 raft_consensus.cc:2454] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:25.971151 10891 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:25.974879 11442 raft_consensus.cc:2781] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:25.975232 11442 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:25.975572 11442 raft_consensus.cc:3037] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:25.985705 11065 tablet_bootstrap.cc:492] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:25.986516 11065 ts_tablet_manager.cc:1397] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.183s user 0.013s sys 0.021s
W20260809 15:51:25.986963 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 67139.8us
I20260809 15:51:25.975747 10930 raft_consensus.cc:2454] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:26.014892 10978 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:26.017616 11348 raft_consensus.cc:2781] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:26.024771 11348 raft_consensus.cc:686] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:26.025194 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:26.037681 11072 tablet_bootstrap.cc:492] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:26.038064 11072 ts_tablet_manager.cc:1397] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.197s user 0.027s sys 0.006s
I20260809 15:51:26.016554 11452 consensus_queue.cc:1035] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.024s
I20260809 15:51:26.042677 11072 raft_consensus.cc:348] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.044116 11072 raft_consensus.cc:374] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:26.044507 11072 raft_consensus.cc:729] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:26.047022 11072 consensus_queue.cc:260] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.025640 11348 consensus_queue.cc:237] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.049678 11388 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:51:26.055713 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 44780.5us
I20260809 15:51:26.063199 11447 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:26.063609 11447 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.065176 11447 leader_election.cc:288] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:26.065984 11383 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:26.066509 11383 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.068938 11383 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:26.071658 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:26.072284 10929 raft_consensus.cc:2454] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:26.072151 11442 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.073683 11072 ts_tablet_manager.cc:1428] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.035s user 0.004s sys 0.003s
I20260809 15:51:26.074791 11072 tablet_bootstrap.cc:492] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:26.075660 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:26.077651 10978 leader_election.cc:302] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:51:26.082949 10891 leader_election.cc:341] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:26.076987 11340 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.087957 11309 raft_consensus.cc:2781] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
W20260809 15:51:26.103621 11457 log.cc:924] Time spent T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.068s user 0.000s sys 0.000s
I20260809 15:51:26.107738 11309 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:26.108314 11309 raft_consensus.cc:3037] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:26.104521 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:26.111253 11016 raft_consensus.cc:3037] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:26.073954 11442 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:26.115204 10893 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:26.116139 10893 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:26.116412 11065 raft_consensus.cc:348] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.124296 11340 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:26.125456 11067 tablet_bootstrap.cc:654] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:26.086997 11018 raft_consensus.cc:2454] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:26.120111 11065 raft_consensus.cc:374] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:26.126665 11065 raft_consensus.cc:729] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:26.127542 11065 consensus_queue.cc:260] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.130113 11016 raft_consensus.cc:2454] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:26.119513 11442 raft_consensus.cc:2781] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:26.130977 11442 raft_consensus.cc:686] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:26.131707 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:26.131783 11442 consensus_queue.cc:237] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.132513 11015 raft_consensus.cc:2379] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 1: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:51:26.133329 11258 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:26.139179 11258 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.141599 11258 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:26.129563 11384 raft_consensus.cc:2726] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:26.144810 10891 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:26.146756 11384 raft_consensus.cc:2781] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:26.148474 11072 tablet_bootstrap.cc:654] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:26.150492 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:26.151012 10929 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:26.152077 11065 ts_tablet_manager.cc:1428] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.165s user 0.015s sys 0.015s
I20260809 15:51:26.154011 11065 tablet_bootstrap.cc:492] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:26.154239 11384 raft_consensus.cc:686] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:26.159443 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:26.158739 11384 consensus_queue.cc:237] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.163384 10733 catalog_manager.cc:5495] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:26.167362 10929 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:26.193691 10811 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:26.198686 11461 raft_consensus.cc:2781] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:26.199156 11461 raft_consensus.cc:686] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:26.200042 11461 consensus_queue.cc:237] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.221050 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:26.219192 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:26.228689 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:26.240147 11067 tablet_bootstrap.cc:492] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:26.240612 11067 ts_tablet_manager.cc:1397] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.298s user 0.021s sys 0.025s
I20260809 15:51:26.241014 10848 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:26.250964 11067 raft_consensus.cc:348] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.251682 11067 raft_consensus.cc:374] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:26.251986 11067 raft_consensus.cc:729] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:26.246933 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:26.257207 11067 consensus_queue.cc:260] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.260456 11072 tablet_bootstrap.cc:492] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:26.268649 11072 ts_tablet_manager.cc:1397] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.194s user 0.022s sys 0.023s
I20260809 15:51:26.260967 11065 tablet_bootstrap.cc:654] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:26.282225 11443 consensus_queue.cc:1035] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
I20260809 15:51:26.284543 11067 ts_tablet_manager.cc:1428] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.044s user 0.006s sys 0.000s
I20260809 15:51:26.285641 11067 tablet_bootstrap.cc:492] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:26.292073 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:26.294327 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:26.295301 11014 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:51:26.299364 10893 leader_election.cc:341] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:26.305526 10809 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:26.307315 10737 catalog_manager.cc:5495] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:26.262506 11309 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.323676 11463 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.005s
W20260809 15:51:26.333122 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 55068.4us
W20260809 15:51:26.337320 10893 leader_election.cc:341] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:26.338213 11016 raft_consensus.cc:3037] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:26.338750 11458 raft_consensus.cc:2726] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:26.344981 10893 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:26.346401 10737 catalog_manager.cc:5495] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
W20260809 15:51:26.350960 10893 leader_election.cc:341] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:26.350987 10978 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:26.354559 11014 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:26.322506 11384 consensus_queue.cc:1035] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.032s
W20260809 15:51:26.363157 10978 leader_election.cc:341] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
W20260809 15:51:26.363804 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 43871.7us
I20260809 15:51:26.349232 11309 leader_election.cc:288] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:26.352906 11383 raft_consensus.cc:2726] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:26.369975 11072 raft_consensus.cc:348] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.371358 11072 raft_consensus.cc:374] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:26.371613 11072 raft_consensus.cc:729] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:26.372166 11072 consensus_queue.cc:260] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.361579 11348 raft_consensus.cc:2726] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:26.377761 11067 tablet_bootstrap.cc:654] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:26.362689 11444 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:26.384747 11072 ts_tablet_manager.cc:1428] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.116s user 0.006s sys 0.001s
I20260809 15:51:26.385478 11072 tablet_bootstrap.cc:492] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:26.386047 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:26.390307 10924 raft_consensus.cc:3037] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:26.380504 11444 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.395388 11444 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:26.402794 10736 catalog_manager.cc:5495] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 0 to 1, leader changed from <none> to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:26.401206 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:26.380154 11460 consensus_queue.cc:1035] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.042s
I20260809 15:51:26.428905 11473 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:26.429572 11473 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.434073 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:26.424749 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:26.439311 11011 raft_consensus.cc:2454] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
W20260809 15:51:26.442584 10978 leader_election.cc:341] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:26.447763 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:26.450364 10978 leader_election.cc:341] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:26.458537 10891 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:26.466903 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:26.474788 11474 raft_consensus.cc:2781] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:26.475425 11474 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:26.475967 11474 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:26.451010 11473 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:51:26.490196 10893 leader_election.cc:341] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
W20260809 15:51:26.494721 10978 leader_election.cc:341] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:26.495262 10978 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:26.498872 11473 raft_consensus.cc:2726] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:26.496955 11239 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:26.508801 11239 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.510028 10840 raft_consensus.cc:3037] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:26.511219 11072 tablet_bootstrap.cc:654] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:26.523404 10924 raft_consensus.cc:2454] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:26.535014 10978 leader_election.cc:302] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:26.542943 11239 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:26.544070 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:26.553642 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:26.555697 10931 raft_consensus.cc:2379] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 1: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:51:26.550876 11447 raft_consensus.cc:2781] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:26.557241 11447 raft_consensus.cc:686] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:26.559386 10811 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:26.561982 11447 consensus_queue.cc:237] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.575256 11067 tablet_bootstrap.cc:492] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:26.575662 11067 ts_tablet_manager.cc:1397] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.290s user 0.007s sys 0.020s
I20260809 15:51:26.583986 11067 raft_consensus.cc:348] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.584709 11067 raft_consensus.cc:374] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:26.584445 11474 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.585071 11067 raft_consensus.cc:729] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:26.586668 11474 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:26.587182 11067 consensus_queue.cc:260] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.591943 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:26.593879 11013 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:26.610234 11067 ts_tablet_manager.cc:1428] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.034s user 0.007s sys 0.001s
I20260809 15:51:26.612010 11067 tablet_bootstrap.cc:492] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:26.613736 11013 raft_consensus.cc:2454] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:26.619649 10891 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:26.620795 11309 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:26.621594 11309 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.626803 11309 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:51:26.624197 11480 log.cc:924] Time spent T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.050s user 0.001s sys 0.001s
I20260809 15:51:26.627472 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:26.631223 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:51:26.631595 10978 leader_election.cc:341] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
W20260809 15:51:26.633795 10978 leader_election.cc:341] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:26.634209 10978 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:26.638234 11309 raft_consensus.cc:2726] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:26.640178 11463 raft_consensus.cc:2726] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:26.643700 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:26.646008 11065 tablet_bootstrap.cc:492] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:26.665126 11065 ts_tablet_manager.cc:1397] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.511s user 0.013s sys 0.015s
W20260809 15:51:26.674156 10789 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 15:51:26.648403 11340 consensus_queue.cc:1035] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:26.646472 11474 raft_consensus.cc:2781] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:26.655618 11309 consensus_queue.cc:1035] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
W20260809 15:51:26.647429 10893 leader_election.cc:341] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:26.677594 11065 raft_consensus.cc:348] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.678397 11065 raft_consensus.cc:374] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:26.678660 11065 raft_consensus.cc:729] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:26.679268 11065 consensus_queue.cc:260] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.681394 11065 ts_tablet_manager.cc:1428] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260809 15:51:26.684209 11474 raft_consensus.cc:686] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:26.692353 11072 tablet_bootstrap.cc:492] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:26.692811 11072 ts_tablet_manager.cc:1397] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.307s user 0.020s sys 0.046s
I20260809 15:51:26.696058 11072 raft_consensus.cc:348] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.696928 11072 raft_consensus.cc:374] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:26.697268 11072 raft_consensus.cc:729] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:26.698100 11072 consensus_queue.cc:260] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.700596 11072 ts_tablet_manager.cc:1428] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260809 15:51:26.701488 11072 tablet_bootstrap.cc:492] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:26.685535 11474 consensus_queue.cc:237] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.706533 11065 tablet_bootstrap.cc:492] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:26.717320 11072 tablet_bootstrap.cc:654] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:26.719451 11065 tablet_bootstrap.cc:654] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
W20260809 15:51:26.725788 10893 consensus_peers.cc:487] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 -> Peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Couldn't send request to peer a53441872c0c416b8e2f26f52399f536. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:26.732492 11072 tablet_bootstrap.cc:492] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:26.732925 11072 ts_tablet_manager.cc:1397] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.032s user 0.007s sys 0.011s
I20260809 15:51:26.674108 11067 tablet_bootstrap.cc:654] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
W20260809 15:51:26.735256 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 44554.1us
I20260809 15:51:26.732882 11475 consensus_queue.cc:1035] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260809 15:51:26.752632 11477 consensus_queue.cc:1035] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260809 15:51:26.763320 11065 tablet_bootstrap.cc:492] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:26.763782 11065 ts_tablet_manager.cc:1397] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.057s user 0.009s sys 0.008s
I20260809 15:51:26.763741 11463 consensus_queue.cc:1035] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:26.776229 11072 raft_consensus.cc:348] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.780223 11072 raft_consensus.cc:374] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:26.780918 11065 raft_consensus.cc:348] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.783640 11065 raft_consensus.cc:374] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:26.783981 11065 raft_consensus.cc:729] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:26.784718 11065 consensus_queue.cc:260] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.786028 11072 raft_consensus.cc:729] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:26.783331 10733 catalog_manager.cc:5495] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:26.786684 11072 consensus_queue.cc:260] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.788913 11346 consensus_queue.cc:1035] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:26.790768 11065 ts_tablet_manager.cc:1428] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.026s user 0.007s sys 0.001s
I20260809 15:51:26.791710 11065 tablet_bootstrap.cc:492] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:26.792461 11384 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:26.793236 11384 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.795928 11072 ts_tablet_manager.cc:1428] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.063s user 0.009s sys 0.001s
I20260809 15:51:26.796995 11072 tablet_bootstrap.cc:492] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:26.796913 11384 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:26.803999 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:26.807507 10893 leader_election.cc:341] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:26.803428 10737 catalog_manager.cc:5495] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:26.834631 11382 consensus_queue.cc:1035] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s
I20260809 15:51:26.839848 11474 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:26.840590 11065 tablet_bootstrap.cc:654] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:26.840370 11474 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.874284 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:26.881685 10893 leader_election.cc:341] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:26.888871 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:26.895393 11458 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:26.896353 11072 tablet_bootstrap.cc:654] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
W20260809 15:51:26.902316 10891 leader_election.cc:341] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:26.902848 10891 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:26.888383 11474 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:26.906587 11346 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:26.896032 11458 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.906677 11443 raft_consensus.cc:2726] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:26.924949 11346 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:26.963406 11458 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:27.038336 11067 tablet_bootstrap.cc:492] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:27.038865 11067 ts_tablet_manager.cc:1397] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.427s user 0.031s sys 0.062s
I20260809 15:51:26.993284 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:51:27.045204 10811 leader_election.cc:341] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:26.983968 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:27.054726 11012 raft_consensus.cc:2454] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:27.016860 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:27.029834 11346 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:27.076335 10809 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:27.077943 11067 raft_consensus.cc:348] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.078683 11067 raft_consensus.cc:374] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:27.078941 11067 raft_consensus.cc:729] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:27.083868 11072 tablet_bootstrap.cc:492] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:27.084313 11072 ts_tablet_manager.cc:1397] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.287s user 0.010s sys 0.015s
I20260809 15:51:27.085496 11067 consensus_queue.cc:260] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.088524 11065 tablet_bootstrap.cc:492] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:27.089200 11065 ts_tablet_manager.cc:1397] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.298s user 0.036s sys 0.055s
I20260809 15:51:27.085336 11463 raft_consensus.cc:2781] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:27.096952 11072 raft_consensus.cc:348] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.099944 11072 raft_consensus.cc:374] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:27.100179 11072 raft_consensus.cc:729] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
W20260809 15:51:27.100759 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 179793us
I20260809 15:51:27.106595 11458 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:27.107131 11458 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.110301 11458 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:51:27.112118 10891 leader_election.cc:341] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:27.112855 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:27.113312 10891 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:27.095916 11463 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:27.114673 11463 raft_consensus.cc:3037] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:27.117622 11067 ts_tablet_manager.cc:1428] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.078s user 0.006s sys 0.002s
I20260809 15:51:27.126955 11382 raft_consensus.cc:2726] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:27.129122 11067 tablet_bootstrap.cc:492] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:27.130450 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:27.131192 11012 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:27.117928 11384 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:27.132017 11384 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.136864 11384 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:27.146586 11067 tablet_bootstrap.cc:654] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:27.146669 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:27.147684 10843 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
W20260809 15:51:27.148746 10811 leader_election.cc:341] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:27.149199 11072 consensus_queue.cc:260] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.152460 11072 ts_tablet_manager.cc:1428] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.068s user 0.001s sys 0.006s
I20260809 15:51:27.153322 11072 tablet_bootstrap.cc:492] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:27.098899 11065 raft_consensus.cc:348] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.161957 11065 raft_consensus.cc:374] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:27.162547 11065 raft_consensus.cc:729] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:27.169533 10978 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:27.172421 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:27.163460 11065 consensus_queue.cc:260] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.176934 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:27.290233 11449 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:27.290798 11449 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:27.291191 11067 tablet_bootstrap.cc:492] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:27.291814 11067 ts_tablet_manager.cc:1397] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.163s user 0.047s sys 0.026s
W20260809 15:51:27.292466 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 121192us
I20260809 15:51:27.295430 11447 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:27.295921 11447 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.296305 11463 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.298061 11463 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:27.297876 11447 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:27.293262 10809 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:27.301635 11458 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:27.301987 11458 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:27.302400 11458 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:51:27.309831 10978 consensus_peers.cc:487] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa -> Peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Couldn't send request to peer a53441872c0c416b8e2f26f52399f536. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:27.310243 11067 raft_consensus.cc:348] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.310132 11458 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.310999 11067 raft_consensus.cc:374] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:27.312107 11067 raft_consensus.cc:729] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:27.314522 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:27.315179 11010 raft_consensus.cc:3037] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:27.314719 11067 consensus_queue.cc:260] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.319262 11072 tablet_bootstrap.cc:654] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:27.291205 11449 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:27.327028 11067 ts_tablet_manager.cc:1428] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.035s user 0.006s sys 0.001s
I20260809 15:51:27.327203 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:27.330618 11065 ts_tablet_manager.cc:1428] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.241s user 0.012s sys 0.004s
I20260809 15:51:27.331493 11065 tablet_bootstrap.cc:492] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:27.336598 11458 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:27.350669 11067 tablet_bootstrap.cc:492] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
W20260809 15:51:27.362046 10978 leader_election.cc:341] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
W20260809 15:51:27.367187 10891 leader_election.cc:341] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:27.370014 11065 tablet_bootstrap.cc:654] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
W20260809 15:51:27.373663 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 51607.6us
I20260809 15:51:27.376770 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:27.371901 11475 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:27.361003 11478 consensus_queue.cc:1035] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260809 15:51:27.381435 11475 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.371917 11505 consensus_queue.cc:1035] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.013s
I20260809 15:51:27.383415 11010 raft_consensus.cc:2454] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:27.384649 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:27.377574 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:27.391304 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:27.383435 11475 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:27.408876 10809 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:51:27.409775 10978 leader_election.cc:341] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:27.410660 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
W20260809 15:51:27.413355 10893 consensus_peers.cc:487] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 -> Peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Couldn't send request to peer a53441872c0c416b8e2f26f52399f536. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: BOOTSTRAPPING. This is attempt 1: this message will repeat every 5th retry.
W20260809 15:51:27.414165 10893 leader_election.cc:341] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:27.414645 10893 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:27.408551 10926 raft_consensus.cc:2454] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:27.419905 11505 raft_consensus.cc:2726] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:27.431843 10978 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:27.420198 11463 raft_consensus.cc:2781] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:27.432667 11463 raft_consensus.cc:686] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:27.433411 11463 consensus_queue.cc:237] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.447173 11072 tablet_bootstrap.cc:492] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:27.447616 11072 ts_tablet_manager.cc:1397] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.294s user 0.012s sys 0.021s
I20260809 15:51:27.449000 11449 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.450176 11017 raft_consensus.cc:2379] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 1: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:51:27.454838 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:27.440579 11309 raft_consensus.cc:2781] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:27.457172 11309 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:27.457598 11309 raft_consensus.cc:3037] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:27.460253 11449 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:51:27.462654 10811 leader_election.cc:341] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:27.469482 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:27.470361 11067 tablet_bootstrap.cc:654] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:27.470152 11017 raft_consensus.cc:2454] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:27.480330 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:27.481074 10843 raft_consensus.cc:2379] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 1: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
W20260809 15:51:27.483333 10893 leader_election.cc:341] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:27.485239 11072 raft_consensus.cc:348] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.485960 11072 raft_consensus.cc:374] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:27.486208 11072 raft_consensus.cc:729] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:27.487748 10891 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:27.486644 11508 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:27.489370 11508 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.486797 11072 consensus_queue.cc:260] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.491326 11508 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:51:27.502907 10811 leader_election.cc:341] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:27.503469 10811 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:27.504784 11508 raft_consensus.cc:2726] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 15:51:27.505868 11072 ts_tablet_manager.cc:1428] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.058s user 0.007s sys 0.000s
I20260809 15:51:27.506740 11072 tablet_bootstrap.cc:492] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:27.506824 11475 raft_consensus.cc:2781] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:27.515520 11475 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:27.516057 11475 raft_consensus.cc:3037] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:27.505640 11309 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.547983 11309 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:27.552836 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:27.560971 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:51:27.565424 10978 leader_election.cc:341] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
W20260809 15:51:27.576594 10811 leader_election.cc:341] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:27.583283 11067 tablet_bootstrap.cc:492] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:27.583796 11067 ts_tablet_manager.cc:1397] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.233s user 0.016s sys 0.051s
I20260809 15:51:27.591588 11065 tablet_bootstrap.cc:492] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:27.592067 11065 ts_tablet_manager.cc:1397] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.261s user 0.013s sys 0.025s
I20260809 15:51:27.594964 11072 tablet_bootstrap.cc:654] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:27.600190 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:27.600926 10927 raft_consensus.cc:3037] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:27.594348 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:27.602880 10930 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:27.607971 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:27.609071 11010 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:27.602344 10736 catalog_manager.cc:5495] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 0 to 1, leader changed from <none> to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:27.615641 10809 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:27.617008 11508 raft_consensus.cc:2781] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:27.617650 11508 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:27.618278 11508 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:27.630625 11065 raft_consensus.cc:348] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.631408 11065 raft_consensus.cc:374] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:27.631659 11065 raft_consensus.cc:729] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:27.632367 11065 consensus_queue.cc:260] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.636277 11065 ts_tablet_manager.cc:1428] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.044s user 0.006s sys 0.002s
I20260809 15:51:27.634325 11475 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.655031 11065 tablet_bootstrap.cc:492] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:27.656384 11475 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:27.642058 10930 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:27.643959 10927 raft_consensus.cc:2454] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:27.658030 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:27.668237 11461 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:27.668872 11461 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:27.672016 10893 leader_election.cc:341] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:27.648005 11067 raft_consensus.cc:348] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.674552 11067 raft_consensus.cc:374] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:27.674892 11067 raft_consensus.cc:729] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:27.645674 11508 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.682983 11067 consensus_queue.cc:260] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.694200 10978 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:27.695194 10978 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:27.706358 11067 ts_tablet_manager.cc:1428] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.122s user 0.004s sys 0.003s
I20260809 15:51:27.702700 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:27.709092 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:27.709887 11012 raft_consensus.cc:3037] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:27.713987 11461 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:27.714783 11067 tablet_bootstrap.cc:492] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:27.703285 11309 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:27.702792 11348 raft_consensus.cc:2781] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
W20260809 15:51:27.725903 10789 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 15:51:27.726472 11348 raft_consensus.cc:686] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:27.739192 11072 tablet_bootstrap.cc:492] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:27.739750 11072 ts_tablet_manager.cc:1397] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.233s user 0.025s sys 0.021s
I20260809 15:51:27.725592 11309 raft_consensus.cc:686] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:27.742194 11309 consensus_queue.cc:237] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.725314 11508 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:27.771659 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:27.774960 10847 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:51:27.775614 10811 leader_election.cc:341] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:27.780922 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:27.781632 11017 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:27.790681 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:27.792238 10809 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:27.730875 11348 consensus_queue.cc:237] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.804394 11012 raft_consensus.cc:2454] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:27.818938 11017 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:27.813977 11072 raft_consensus.cc:348] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.819509 11072 raft_consensus.cc:374] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:27.819890 11072 raft_consensus.cc:729] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:27.820159 10809 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:27.820731 11072 consensus_queue.cc:260] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.822692 11508 raft_consensus.cc:2781] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:27.827477 11508 raft_consensus.cc:686] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:27.828323 11508 consensus_queue.cc:237] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.830659 10891 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:27.834723 11065 tablet_bootstrap.cc:654] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:27.845332 10737 catalog_manager.cc:5495] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:27.857838 10737 catalog_manager.cc:5495] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:27.868099 11072 ts_tablet_manager.cc:1428] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.128s user 0.010s sys 0.002s
I20260809 15:51:27.853287 11384 raft_consensus.cc:2781] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:27.869499 11072 tablet_bootstrap.cc:492] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:27.874264 11067 tablet_bootstrap.cc:654] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:27.890102 11309 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:27.890735 11309 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.872392 11384 raft_consensus.cc:686] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:27.895774 11309 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:27.894556 11384 consensus_queue.cc:237] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:27.896871 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:27.897949 10927 raft_consensus.cc:2454] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:27.895313 11463 consensus_queue.cc:1035] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.007s
I20260809 15:51:27.954602 11065 tablet_bootstrap.cc:492] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:27.956837 11065 ts_tablet_manager.cc:1397] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.302s user 0.019s sys 0.011s
I20260809 15:51:27.958817 10978 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:27.964354 11309 raft_consensus.cc:2781] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:27.965035 11309 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:27.965521 11309 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:27.975212 11072 tablet_bootstrap.cc:654] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:28.001685 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:28.007812 10811 consensus_peers.cc:487] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 -> Peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Couldn't send request to peer 20730d75ad954a82b126148f8152a9f3. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:28.032155 11067 tablet_bootstrap.cc:492] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:28.032871 11067 ts_tablet_manager.cc:1397] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.318s user 0.013s sys 0.023s
I20260809 15:51:28.040973 11067 raft_consensus.cc:348] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.052747 11067 raft_consensus.cc:374] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:28.053071 11067 raft_consensus.cc:729] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
W20260809 15:51:28.049800 10978 leader_election.cc:341] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:28.050393 11065 raft_consensus.cc:348] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.055385 11065 raft_consensus.cc:374] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:28.055666 11065 raft_consensus.cc:729] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:28.056294 11065 consensus_queue.cc:260] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.058368 11065 ts_tablet_manager.cc:1428] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.101s user 0.006s sys 0.002s
I20260809 15:51:28.059837 11065 tablet_bootstrap.cc:492] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:28.053673 11067 consensus_queue.cc:260] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.123881 11067 ts_tablet_manager.cc:1428] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.091s user 0.008s sys 0.005s
I20260809 15:51:28.129057 10736 catalog_manager.cc:5495] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:28.138367 11072 tablet_bootstrap.cc:492] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:28.138701 11067 tablet_bootstrap.cc:492] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:28.140473 11072 ts_tablet_manager.cc:1397] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.271s user 0.021s sys 0.012s
I20260809 15:51:28.133916 11309 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.149303 11309 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:28.150743 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:28.151580 10927 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:28.154587 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:51:28.159449 10978 leader_election.cc:341] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:28.172183 11239 raft_consensus.cc:2726] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:28.187430 11447 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:28.192822 11348 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:28.193367 11348 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.194826 11072 raft_consensus.cc:348] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.195534 11348 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:28.195674 11072 raft_consensus.cc:374] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:28.195996 11072 raft_consensus.cc:729] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:28.196607 11072 consensus_queue.cc:260] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.200580 10927 raft_consensus.cc:2454] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:28.202145 11065 tablet_bootstrap.cc:654] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:28.205369 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:51:28.214939 10978 leader_election.cc:341] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:28.235026 11072 ts_tablet_manager.cc:1428] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.094s user 0.003s sys 0.004s
I20260809 15:51:28.236348 11072 tablet_bootstrap.cc:492] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:28.235992 11309 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:28.241978 10978 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:28.245735 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:28.247361 10927 raft_consensus.cc:2454] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:28.247490 11309 raft_consensus.cc:2781] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:28.258412 11309 raft_consensus.cc:686] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:28.259481 11309 consensus_queue.cc:237] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.262431 11067 tablet_bootstrap.cc:654] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:28.271803 10978 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:28.288025 11520 raft_consensus.cc:2781] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:28.288420 11520 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:28.288764 11520 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:28.344698 11072 tablet_bootstrap.cc:654] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:28.346895 11065 tablet_bootstrap.cc:492] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:28.347409 11065 ts_tablet_manager.cc:1397] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.288s user 0.015s sys 0.012s
I20260809 15:51:28.350311 11065 raft_consensus.cc:348] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.350039 10737 catalog_manager.cc:5495] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 0 to 1, leader changed from <none> to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:28.351075 11065 raft_consensus.cc:374] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:28.351402 11065 raft_consensus.cc:729] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:28.352016 11065 consensus_queue.cc:260] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:28.352440 10978 consensus_peers.cc:487] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa -> Peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Couldn't send request to peer a53441872c0c416b8e2f26f52399f536. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:28.358472 10931 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:28.364575 11067 tablet_bootstrap.cc:492] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:28.365023 11067 ts_tablet_manager.cc:1397] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.227s user 0.009s sys 0.011s
I20260809 15:51:28.366953 11067 raft_consensus.cc:348] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.367506 11067 raft_consensus.cc:374] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:28.367708 11067 raft_consensus.cc:729] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:28.368170 11067 consensus_queue.cc:260] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.369691 11067 ts_tablet_manager.cc:1428] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 15:51:28.388602 11463 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:28.391608 11065 ts_tablet_manager.cc:1428] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.044s user 0.006s sys 0.001s
I20260809 15:51:28.392686 11065 tablet_bootstrap.cc:492] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:28.355249 11520 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.401402 11443 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:28.401919 11443 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.402570 11520 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:28.405521 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:28.406361 11067 tablet_bootstrap.cc:492] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:28.406324 11012 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:28.407987 10891 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:28.413901 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:28.414636 10927 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:28.428051 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:28.416810 11443 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:28.440472 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:51:28.442271 10978 leader_election.cc:341] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:28.444690 11505 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:28.446556 11505 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.449986 11515 raft_consensus.cc:2781] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:51:28.452188 11515 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:51:28.450929 11309 consensus_queue.cc:1035] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.002s
I20260809 15:51:28.450462 11447 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:28.453866 11505 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:28.453687 11447 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.455988 11447 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:28.457290 11065 tablet_bootstrap.cc:654] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:28.458177 10733 catalog_manager.cc:5495] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:28.455511 11515 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:28.463137 11529 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:28.463236 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:28.464599 11072 tablet_bootstrap.cc:492] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:28.466560 11067 tablet_bootstrap.cc:654] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:28.466450 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:28.470768 11516 consensus_queue.cc:1035] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:28.467941 10927 raft_consensus.cc:2454] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:28.487313 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:28.496657 11463 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.006s
I20260809 15:51:28.501842 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:28.536712 10931 raft_consensus.cc:2454] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:28.508374 11529 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.538391 11072 ts_tablet_manager.cc:1397] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.302s user 0.027s sys 0.035s
I20260809 15:51:28.540182 11529 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:28.524093 11515 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.541683 10844 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:28.543414 10926 raft_consensus.cc:1226] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 1. Current term is 2. Ops: []
I20260809 15:51:28.548066 11072 raft_consensus.cc:348] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.548736 11072 raft_consensus.cc:374] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:28.548980 11072 raft_consensus.cc:729] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:28.549496 11072 consensus_queue.cc:260] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:28.550627 10891 leader_election.cc:341] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
W20260809 15:51:28.550861 10893 consensus_peers.cc:487] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 -> Peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Couldn't send request to peer a53441872c0c416b8e2f26f52399f536. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
W20260809 15:51:28.551312 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 50010.2us
I20260809 15:51:28.554215 10978 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:28.554986 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:28.555174 11473 raft_consensus.cc:2781] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:28.555671 11473 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:28.555712 10926 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 0.
I20260809 15:51:28.556169 11473 raft_consensus.cc:3037] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:28.558281 10978 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:28.559867 11309 raft_consensus.cc:2781] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:28.560338 11309 raft_consensus.cc:686] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:28.561060 10811 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:28.561097 11309 consensus_queue.cc:237] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.562667 11508 raft_consensus.cc:2781] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:28.563066 11508 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:28.563458 11508 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:28.562810 11515 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:28.566629 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:28.567163 11017 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:28.571633 11065 tablet_bootstrap.cc:492] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:28.572141 11065 ts_tablet_manager.cc:1397] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.180s user 0.008s sys 0.008s
I20260809 15:51:28.572389 11072 ts_tablet_manager.cc:1428] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.033s user 0.004s sys 0.000s
I20260809 15:51:28.573376 11072 tablet_bootstrap.cc:492] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:28.574959 11065 raft_consensus.cc:348] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.575692 11065 raft_consensus.cc:374] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:28.576040 11065 raft_consensus.cc:729] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:28.576674 11065 consensus_queue.cc:260] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.584004 11067 tablet_bootstrap.cc:492] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:28.584676 11067 ts_tablet_manager.cc:1397] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.179s user 0.012s sys 0.008s
I20260809 15:51:28.594180 11067 raft_consensus.cc:348] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.594856 11067 raft_consensus.cc:374] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:28.560119 11529 consensus_queue.cc:1046] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:28.595348 11065 ts_tablet_manager.cc:1428] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.023s user 0.004s sys 0.004s
I20260809 15:51:28.597810 11065 tablet_bootstrap.cc:492] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:28.595109 11067 raft_consensus.cc:729] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:28.607210 11067 consensus_queue.cc:260] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.608642 11067 ts_tablet_manager.cc:1428] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.024s user 0.003s sys 0.000s
I20260809 15:51:28.596652 11529 raft_consensus.cc:3032] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:28.613044 11072 tablet_bootstrap.cc:654] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:28.613435 11017 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:51:28.619304 11529 raft_consensus.cc:729] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:28.617816 11461 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:28.620086 11010 raft_consensus.cc:1226] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 1. Current term is 2. Ops: [1.1-1.1]
I20260809 15:51:28.620074 11461 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.622798 11067 tablet_bootstrap.cc:492] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:28.623510 11461 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:28.628852 11067 tablet_bootstrap.cc:654] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:28.633749 10891 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:28.634027 11447 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:28.634508 11516 raft_consensus.cc:2781] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:28.634560 11447 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.635182 11516 raft_consensus.cc:686] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:28.634429 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:28.640673 11065 tablet_bootstrap.cc:654] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:28.636086 11516 consensus_queue.cc:237] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.642967 11508 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.646497 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:28.647094 10931 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:28.650852 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:28.652241 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:28.654326 11067 tablet_bootstrap.cc:492] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:28.654796 11067 ts_tablet_manager.cc:1397] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.032s user 0.011s sys 0.009s
W20260809 15:51:28.655620 10978 leader_election.cc:341] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:28.655967 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:28.653059 11508 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:28.657939 10893 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:28.659664 11067 raft_consensus.cc:348] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.660334 11067 raft_consensus.cc:374] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:28.660609 11067 raft_consensus.cc:729] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:28.660460 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:28.660745 11529 consensus_queue.cc:260] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.661193 10926 raft_consensus.cc:2454] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 0.
I20260809 15:51:28.661826 11529 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:28.664085 11067 consensus_queue.cc:260] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.636648 11447 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:28.665766 11067 ts_tablet_manager.cc:1428] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.011s user 0.005s sys 0.000s
I20260809 15:51:28.666062 11072 tablet_bootstrap.cc:492] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:28.666486 11072 ts_tablet_manager.cc:1397] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.093s user 0.012s sys 0.007s
I20260809 15:51:28.668385 10978 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:28.669127 11072 raft_consensus.cc:348] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.669852 11072 raft_consensus.cc:374] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:28.670228 11072 raft_consensus.cc:729] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:28.670156 11447 consensus_queue.cc:1035] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:28.669787 11309 raft_consensus.cc:2781] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:28.671073 11309 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:28.670920 11072 consensus_queue.cc:260] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.671397 11309 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:28.673349 11072 ts_tablet_manager.cc:1428] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260809 15:51:28.674371 11072 tablet_bootstrap.cc:492] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3: Bootstrap starting.
I20260809 15:51:28.640558 11473 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.672469 10736 catalog_manager.cc:5495] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 1 to 2, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:28.676033 11473 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:28.678555 10931 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:28.679558 11516 raft_consensus.cc:2741] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260809 15:51:28.679853 11072 tablet_bootstrap.cc:654] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:28.679682 10811 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:28.682016 11508 raft_consensus.cc:2781] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:28.682700 11508 raft_consensus.cc:686] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:28.683525 11508 consensus_queue.cc:237] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.678586 11309 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.688607 11309 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:28.692349 11072 tablet_bootstrap.cc:492] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3: No bootstrap required, opened a new log
I20260809 15:51:28.692785 11072 ts_tablet_manager.cc:1397] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3: Time spent bootstrapping tablet: real 0.019s user 0.011s sys 0.003s
I20260809 15:51:28.695485 11067 tablet_bootstrap.cc:492] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa: Bootstrap starting.
I20260809 15:51:28.697299 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:28.697870 10927 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:28.698086 11072 raft_consensus.cc:348] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.698835 11072 raft_consensus.cc:374] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:28.698787 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:28.699198 11072 raft_consensus.cc:729] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Initialized, Role: FOLLOWER
I20260809 15:51:28.699436 10931 raft_consensus.cc:3037] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:28.699750 11072 consensus_queue.cc:260] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.702207 11072 ts_tablet_manager.cc:1428] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260809 15:51:28.740118 11065 tablet_bootstrap.cc:492] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:28.758469 10931 raft_consensus.cc:2454] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:28.770723 11065 ts_tablet_manager.cc:1397] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.173s user 0.013s sys 0.001s
W20260809 15:51:28.763788 10809 leader_election.cc:341] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:28.745049 10927 raft_consensus.cc:2454] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:51:28.740979 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:28.766640 10733 catalog_manager.cc:5495] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 0 to 1, leader changed from <none> to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:28.776643 10736 catalog_manager.cc:5495] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:28.750041 11067 tablet_bootstrap.cc:654] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:28.781773 10842 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:51:28.790588 10927 raft_consensus.cc:3037] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:28.793155 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:28.794889 11447 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:28.795356 11447 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.797212 11447 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:28.799386 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:28.799834 10841 raft_consensus.cc:3037] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:28.802615 11065 raft_consensus.cc:348] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.803304 11065 raft_consensus.cc:374] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:28.803575 11065 raft_consensus.cc:729] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:28.807242 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:28.808043 10930 raft_consensus.cc:2379] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 1: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:51:28.808848 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:28.809391 11017 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
W20260809 15:51:28.817740 10978 leader_election.cc:341] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:28.820289 10978 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:28.820783 11017 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:51:28.824013 11447 raft_consensus.cc:2781] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:28.822754 11065 consensus_queue.cc:260] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.824105 10978 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:28.824551 11447 raft_consensus.cc:686] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:28.830433 11447 consensus_queue.cc:237] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.833958 11348 raft_consensus.cc:2781] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:28.834491 11348 raft_consensus.cc:686] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:28.835278 11348 consensus_queue.cc:237] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.840086 11065 ts_tablet_manager.cc:1428] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.040s user 0.004s sys 0.000s
I20260809 15:51:28.847147 10809 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:51:28.853860 10978 leader_election.cc:341] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
W20260809 15:51:28.857290 10978 consensus_peers.cc:487] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa -> Peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Couldn't send request to peer a53441872c0c416b8e2f26f52399f536. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: INITIALIZED. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:28.859753 11065 tablet_bootstrap.cc:492] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:28.861550 11067 tablet_bootstrap.cc:492] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa: No bootstrap required, opened a new log
I20260809 15:51:28.862043 11067 ts_tablet_manager.cc:1397] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa: Time spent bootstrapping tablet: real 0.167s user 0.011s sys 0.003s
W20260809 15:51:28.862957 10978 leader_election.cc:341] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:28.866855 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:28.868396 11067 raft_consensus.cc:348] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.869071 11067 raft_consensus.cc:374] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:28.869330 11067 raft_consensus.cc:729] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Initialized, Role: FOLLOWER
I20260809 15:51:28.871783 10978 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:28.873085 11520 raft_consensus.cc:2726] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:28.873364 10842 raft_consensus.cc:3037] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:28.869961 11067 consensus_queue.cc:260] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.884475 11067 ts_tablet_manager.cc:1428] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa: Time spent starting tablet: real 0.022s user 0.006s sys 0.000s
I20260809 15:51:28.896461 11065 tablet_bootstrap.cc:654] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
W20260809 15:51:28.934284 10789 auto_rebalancer.cc:227] Could not retrieve cluster info: Service unavailable: Tablet not running
I20260809 15:51:28.942457 11529 raft_consensus.cc:2726] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:28.962504 11065 tablet_bootstrap.cc:492] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:28.962896 11065 ts_tablet_manager.cc:1397] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.111s user 0.010s sys 0.013s
I20260809 15:51:28.964512 11065 raft_consensus.cc:348] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.965075 11065 raft_consensus.cc:374] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:28.965327 11065 raft_consensus.cc:729] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:28.965869 11065 consensus_queue.cc:260] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.968576 11505 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:28.969070 11505 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:28.970937 11505 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:28.972756 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:28.973400 11017 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 0.
I20260809 15:51:28.977324 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:28.977914 11065 ts_tablet_manager.cc:1428] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260809 15:51:28.978927 11065 tablet_bootstrap.cc:492] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:28.979918 10891 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:28.980938 11505 raft_consensus.cc:2781] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 15:51:28.981266 11505 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 15:51:28.982299 10893 leader_election.cc:341] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:28.981587 11505 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:28.970947 10733 catalog_manager.cc:5495] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:28.984757 10733 catalog_manager.cc:5495] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 0 to 1, leader changed from <none> to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:28.997957 11065 tablet_bootstrap.cc:654] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:29.013180 11065 tablet_bootstrap.cc:492] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:29.013603 11065 ts_tablet_manager.cc:1397] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.035s user 0.005s sys 0.009s
I20260809 15:51:29.015792 11065 raft_consensus.cc:348] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:29.016304 11065 raft_consensus.cc:374] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:29.016525 11065 raft_consensus.cc:729] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:29.017086 11065 consensus_queue.cc:260] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:29.018996 11065 ts_tablet_manager.cc:1428] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260809 15:51:29.019425 11505 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:29.020843 11529 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:29.021368 11529 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:29.022336 11505 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:29.023617 11065 tablet_bootstrap.cc:492] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:29.024041 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:29.024864 10923 raft_consensus.cc:2379] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 1: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:51:29.028147 11529 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:29.028825 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:29.030200 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:29.030566 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
W20260809 15:51:29.031384 10893 leader_election.cc:341] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): Illegal state: must be running to vote when last-logged opid is not known
I20260809 15:51:29.032505 11012 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:29.032001 10809 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:29.034557 11529 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:29.038772 11012 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:29.039693 10891 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:51:29.040644 11505 raft_consensus.cc:2781] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 15:51:29.041112 11505 raft_consensus.cc:686] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:29.041889 11505 consensus_queue.cc:237] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:29.044088 11447 consensus_queue.cc:1035] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:29.046458 11065 tablet_bootstrap.cc:654] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:29.053781 11529 raft_consensus.cc:2726] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:29.070297 10736 catalog_manager.cc:5495] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 0 to 1, leader changed from <none> to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:29.076943 11065 tablet_bootstrap.cc:492] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:29.077414 11065 ts_tablet_manager.cc:1397] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.054s user 0.007s sys 0.005s
I20260809 15:51:29.079587 11521 consensus_queue.cc:1035] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:29.082845 11065 raft_consensus.cc:348] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:29.083491 11065 raft_consensus.cc:374] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:29.083702 11065 raft_consensus.cc:729] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:29.084317 11065 consensus_queue.cc:260] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:29.103654 11065 ts_tablet_manager.cc:1428] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.026s user 0.007s sys 0.000s
I20260809 15:51:29.104844 11065 tablet_bootstrap.cc:492] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:29.111842 11065 tablet_bootstrap.cc:654] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:29.139882 11065 tablet_bootstrap.cc:492] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:29.140353 11065 ts_tablet_manager.cc:1397] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.036s user 0.013s sys 0.011s
I20260809 15:51:29.142803 11065 raft_consensus.cc:348] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:29.143532 11065 raft_consensus.cc:374] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:29.143740 11065 raft_consensus.cc:729] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:29.144270 11065 consensus_queue.cc:260] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:29.151192 11065 ts_tablet_manager.cc:1428] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.011s user 0.007s sys 0.001s
I20260809 15:51:29.152280 11065 tablet_bootstrap.cc:492] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:29.155926 10842 raft_consensus.cc:3037] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:29.157750 11065 tablet_bootstrap.cc:654] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:29.164315 11529 consensus_queue.cc:1035] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:29.175024 11529 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:29.175459 11529 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:29.177281 11529 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:29.179340 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:29.184576 11065 tablet_bootstrap.cc:492] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:29.185089 11065 ts_tablet_manager.cc:1397] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.033s user 0.017s sys 0.010s
I20260809 15:51:29.188779 11065 raft_consensus.cc:348] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:29.189512 11065 raft_consensus.cc:374] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:29.189776 11065 raft_consensus.cc:729] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:29.190353 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:29.190507 11065 consensus_queue.cc:260] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:29.194123 11065 ts_tablet_manager.cc:1428] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.008s user 0.002s sys 0.004s
I20260809 15:51:29.195242 11065 tablet_bootstrap.cc:492] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536: Bootstrap starting.
I20260809 15:51:29.203970 11065 tablet_bootstrap.cc:654] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536: Neither blocks nor log segments found. Creating new log.
I20260809 15:51:29.205003 11012 raft_consensus.cc:1261] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:29.206615 11516 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:29.221359 10809 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:29.222108 11529 raft_consensus.cc:2726] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:29.224068 11507 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:29.224509 11507 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:29.226524 11507 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:29.227419 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:29.228996 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:29.230021 10927 raft_consensus.cc:2379] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 1: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:51:29.231036 10811 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:29.231467 11507 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:29.232115 11529 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:29.231873 11507 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:29.233843 11507 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:29.239002 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:29.240270 10923 raft_consensus.cc:2379] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 1: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:51:29.235800 11065 tablet_bootstrap.cc:492] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536: No bootstrap required, opened a new log
I20260809 15:51:29.236826 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:29.238616 10844 raft_consensus.cc:1261] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:29.248082 10809 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:29.242406 11065 ts_tablet_manager.cc:1397] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536: Time spent bootstrapping tablet: real 0.047s user 0.021s sys 0.000s
I20260809 15:51:29.244658 11516 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:29.249398 11508 raft_consensus.cc:3037] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:29.250880 11507 consensus_queue.cc:1035] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:29.263413 11065 raft_consensus.cc:348] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:29.263999 10848 raft_consensus.cc:3037] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:29.264158 11065 raft_consensus.cc:374] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 15:51:29.264475 11065 raft_consensus.cc:729] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Initialized, Role: FOLLOWER
I20260809 15:51:29.270758 11065 consensus_queue.cc:260] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:29.272966 11508 raft_consensus.cc:2726] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:29.284981 11065 ts_tablet_manager.cc:1428] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536: Time spent starting tablet: real 0.036s user 0.006s sys 0.000s
I20260809 15:51:29.288990 11473 consensus_queue.cc:1035] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:29.307449 11348 consensus_queue.cc:1035] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:29.281908 11529 raft_consensus.cc:2726] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 15:51:29.325022 11447 consensus_queue.cc:1035] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:29.358736 10841 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:29.376986 11309 consensus_queue.cc:1035] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 15:51:29.463150 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 61151.7us
I20260809 15:51:29.484520 11382 consensus_queue.cc:1035] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 15:51:29.487646 10842 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 0 FOLLOWER]: Advancing to term 1
I20260809 15:51:29.498044 11382 consensus_queue.cc:1035] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 15:51:29.766523 9552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 15:51:29.808718 11593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:51:29.809700 11596 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 15:51:29.810418 11594 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 15:51:29.812813 9552 server_base.cc:1034] running on GCE node
I20260809 15:51:29.815058 9552 hybrid_clock.cc:584] initializing the hybrid clock with 'system_unsync' time source
W20260809 15:51:29.815310 9552 system_unsync_time.cc:38] NTP support is disabled. Clock error bounds will not be accurate. This configuration is not suitable for distributed clusters.
I20260809 15:51:29.815501 9552 hybrid_clock.cc:648] HybridClock initialized: now 1786290689815481 us; error 0 us; skew 500 ppm
I20260809 15:51:29.816040 9552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 15:51:29.828208 9552 webserver.cc:460] Webserver started at http://127.9.84.4:34089/ using document root <none> and password file <none>
I20260809 15:51:29.829051 9552 fs_manager.cc:359] Metadata directory not provided
I20260809 15:51:29.829376 9552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 15:51:29.830489 9552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 15:51:29.833570 9552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-3-root/instance:
uuid: "c7aa762318cb4e97875333f10679a9f9"
format_stamp: "Formatted at 2026-08-09 15:51:29 on dist-test-slave-mn4r"
I20260809 15:51:29.841337 9552 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.002s
I20260809 15:51:29.847358 11601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 15:51:29.848222 9552 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260809 15:51:29.848511 9552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-3-root
uuid: "c7aa762318cb4e97875333f10679a9f9"
format_stamp: "Formatted at 2026-08-09 15:51:29 on dist-test-slave-mn4r"
I20260809 15:51:29.848779 9552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-3-root
metadata directory: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-3-root
1 data directories: /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-3-root/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 15:51:29.890547 9552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 15:51:29.892009 9552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 15:51:29.893862 9552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 15:51:29.896767 9552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 15:51:29.896931 9552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 15:51:29.897229 9552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 15:51:29.897395 9552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 15:51:30.084661 9552 rpc_server.cc:307] RPC server started. Bound to: 127.9.84.4:37685
I20260809 15:51:30.084837 11667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.84.4:37685 every 8 connection(s)
I20260809 15:51:30.091267 9552 auto_leader_rebalancer-test.cc:413] add a tserver: c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:30.111271 11668 heartbeater.cc:347] Connected to a master server at 127.9.84.62:42773
I20260809 15:51:30.111769 11668 heartbeater.cc:464] Registering TS with master...
I20260809 15:51:30.112671 11668 heartbeater.cc:511] Master 127.9.84.62:42773 requested a full tablet report, sending...
I20260809 15:51:30.114713 10736 ts_manager.cc:194] Registered new tserver with Master: c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:51:30.116631 10736 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:59504
I20260809 15:51:31.116269 11011 consensus_queue.cc:237] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:31.119565 11668 heartbeater.cc:503] Master 127.9.84.62:42773 was elected leader, sending a full tablet report...
I20260809 15:51:31.134958 10841 raft_consensus.cc:1261] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:31.133963 10931 raft_consensus.cc:1261] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:31.136475 11589 consensus_queue.cc:1035] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:31.138049 11520 consensus_queue.cc:1035] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:31.167539 11520 raft_consensus.cc:2932] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:31.177424 10931 raft_consensus.cc:2932] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:31.188809 10736 catalog_manager.cc:5495] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:31.192291 11011 consensus_queue.cc:237] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
W20260809 15:51:31.203229 10978 consensus_peers.cc:487] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0b9fb9f88a6847af87fef710d1a2793b. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:31.205905 10841 raft_consensus.cc:2932] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
W20260809 15:51:31.228756 10978 consensus_peers.cc:487] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: f866570ff0c5456ba186b9234cb692c8. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:31.230167 10842 raft_consensus.cc:1261] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:31.234790 11680 consensus_queue.cc:1035] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:31.229992 10931 raft_consensus.cc:1261] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:31.239851 11680 consensus_queue.cc:1035] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:31.270685 10844 raft_consensus.cc:2932] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:31.274613 11520 raft_consensus.cc:2932] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:31.309530 10737 catalog_manager.cc:5495] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:31.316903 10931 raft_consensus.cc:2932] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:31.368532 10841 consensus_queue.cc:237] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:31.376117 11011 raft_consensus.cc:1261] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:31.378317 11684 consensus_queue.cc:1035] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:31.386243 11684 raft_consensus.cc:2932] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:31.388016 10931 raft_consensus.cc:1261] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:31.389452 11529 consensus_queue.cc:1035] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:31.390331 11011 raft_consensus.cc:2932] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:31.404712 10931 raft_consensus.cc:2932] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:31.416450 10737 catalog_manager.cc:5495] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
W20260809 15:51:31.432401 10811 consensus_peers.cc:487] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: dd8bcd39f37d403f9d8d8f43d870a878. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:31.457455 10927 consensus_queue.cc:237] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:31.464638 11011 raft_consensus.cc:1261] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:31.466210 11687 consensus_queue.cc:1035] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:31.474117 10841 raft_consensus.cc:1261] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:31.476473 11687 consensus_queue.cc:1035] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260809 15:51:31.484743 10893 consensus_peers.cc:487] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b6ba8250f4ef43bfbf8f517e86a131a1. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:31.484972 11687 raft_consensus.cc:2932] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:31.495373 10841 raft_consensus.cc:2932] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:31.500845 11010 raft_consensus.cc:2932] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:31.508770 10737 catalog_manager.cc:5495] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:31.593794 11704 ts_tablet_manager.cc:927] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:31.603046 11704 tablet_copy_client.cc:316] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.3:39223
I20260809 15:51:31.631690 11028 tablet_copy_service.cc:140] P d74944fa9fa14ec78b0640ea709ff3aa: Received BeginTabletCopySession request for tablet 0b9fb9f88a6847af87fef710d1a2793b from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:39978)
I20260809 15:51:31.632377 11028 tablet_copy_service.cc:161] P d74944fa9fa14ec78b0640ea709ff3aa: Beginning new tablet copy session on tablet 0b9fb9f88a6847af87fef710d1a2793b from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:39978: session id = c7aa762318cb4e97875333f10679a9f9-0b9fb9f88a6847af87fef710d1a2793b
I20260809 15:51:31.642269 11028 tablet_copy_source_session.cc:205] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:31.647862 11704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b9fb9f88a6847af87fef710d1a2793b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:31.648872 11708 ts_tablet_manager.cc:927] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:31.674656 11708 tablet_copy_client.cc:316] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.3:39223
I20260809 15:51:31.677281 11028 tablet_copy_service.cc:140] P d74944fa9fa14ec78b0640ea709ff3aa: Received BeginTabletCopySession request for tablet f866570ff0c5456ba186b9234cb692c8 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:39978)
I20260809 15:51:31.677774 11028 tablet_copy_service.cc:161] P d74944fa9fa14ec78b0640ea709ff3aa: Beginning new tablet copy session on tablet f866570ff0c5456ba186b9234cb692c8 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:39978: session id = c7aa762318cb4e97875333f10679a9f9-f866570ff0c5456ba186b9234cb692c8
I20260809 15:51:31.681941 11704 tablet_copy_client.cc:791] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:31.686960 11704 tablet_copy_client.cc:655] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:31.693338 11704 tablet_copy_client.cc:531] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:31.702934 11704 tablet_bootstrap.cc:492] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:31.720906 11028 tablet_copy_source_session.cc:205] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:31.726567 11708 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f866570ff0c5456ba186b9234cb692c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:31.760910 11704 tablet_bootstrap.cc:492] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:31.761950 11704 tablet_bootstrap.cc:492] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:31.762598 11704 ts_tablet_manager.cc:1397] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.060s user 0.015s sys 0.017s
I20260809 15:51:31.766960 11708 tablet_copy_client.cc:791] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:31.767526 11708 tablet_copy_client.cc:655] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:31.765362 11704 raft_consensus.cc:348] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:31.768049 11704 raft_consensus.cc:729] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:31.768694 11704 consensus_queue.cc:260] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:31.780095 11704 ts_tablet_manager.cc:1428] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.017s user 0.004s sys 0.007s
I20260809 15:51:31.780158 11708 tablet_copy_client.cc:531] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:31.783478 11028 tablet_copy_service.cc:342] P d74944fa9fa14ec78b0640ea709ff3aa: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-0b9fb9f88a6847af87fef710d1a2793b received from {username='slave'} at 127.0.0.1:39978
I20260809 15:51:31.783932 11028 tablet_copy_service.cc:433] P d74944fa9fa14ec78b0640ea709ff3aa: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-0b9fb9f88a6847af87fef710d1a2793b on tablet 0b9fb9f88a6847af87fef710d1a2793b with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:31.789837 11708 tablet_bootstrap.cc:492] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:31.816337 11704 ts_tablet_manager.cc:927] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615)
I20260809 15:51:31.820655 11708 tablet_bootstrap.cc:492] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:31.821563 11708 tablet_bootstrap.cc:492] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:31.822278 11708 ts_tablet_manager.cc:1397] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.033s user 0.022s sys 0.004s
I20260809 15:51:31.832875 11704 tablet_copy_client.cc:316] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.1:46615
I20260809 15:51:31.825538 11708 raft_consensus.cc:348] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:31.835983 11708 raft_consensus.cc:729] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:31.836658 11708 consensus_queue.cc:260] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:31.863528 10858 tablet_copy_service.cc:140] P a53441872c0c416b8e2f26f52399f536: Received BeginTabletCopySession request for tablet dd8bcd39f37d403f9d8d8f43d870a878 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:54342)
I20260809 15:51:31.864161 10858 tablet_copy_service.cc:161] P a53441872c0c416b8e2f26f52399f536: Beginning new tablet copy session on tablet dd8bcd39f37d403f9d8d8f43d870a878 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:54342: session id = c7aa762318cb4e97875333f10679a9f9-dd8bcd39f37d403f9d8d8f43d870a878
I20260809 15:51:31.865721 11708 ts_tablet_manager.cc:1428] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.043s user 0.006s sys 0.000s
I20260809 15:51:31.870646 11028 tablet_copy_service.cc:342] P d74944fa9fa14ec78b0640ea709ff3aa: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-f866570ff0c5456ba186b9234cb692c8 received from {username='slave'} at 127.0.0.1:39978
I20260809 15:51:31.871057 11028 tablet_copy_service.cc:433] P d74944fa9fa14ec78b0640ea709ff3aa: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-f866570ff0c5456ba186b9234cb692c8 on tablet f866570ff0c5456ba186b9234cb692c8 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:31.872418 10858 tablet_copy_source_session.cc:205] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:31.875759 11704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dd8bcd39f37d403f9d8d8f43d870a878. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:31.895859 11704 tablet_copy_client.cc:791] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:31.897914 11704 tablet_copy_client.cc:655] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:31.901443 11704 tablet_copy_client.cc:531] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:31.911610 11704 tablet_bootstrap.cc:492] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:31.932735 11704 tablet_bootstrap.cc:492] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:31.933565 11704 tablet_bootstrap.cc:492] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:31.934746 11704 ts_tablet_manager.cc:1397] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.023s user 0.017s sys 0.004s
I20260809 15:51:31.937346 11704 raft_consensus.cc:348] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:31.938233 11704 raft_consensus.cc:729] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:31.938808 11704 consensus_queue.cc:260] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:31.940934 11704 ts_tablet_manager.cc:1428] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260809 15:51:31.942554 10858 tablet_copy_service.cc:342] P a53441872c0c416b8e2f26f52399f536: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-dd8bcd39f37d403f9d8d8f43d870a878 received from {username='slave'} at 127.0.0.1:54342
I20260809 15:51:31.942926 10858 tablet_copy_service.cc:433] P a53441872c0c416b8e2f26f52399f536: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-dd8bcd39f37d403f9d8d8f43d870a878 on tablet dd8bcd39f37d403f9d8d8f43d870a878 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:32.019078 11704 ts_tablet_manager.cc:927] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:32.021237 11704 tablet_copy_client.cc:316] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.2:32783
I20260809 15:51:32.035861 10942 tablet_copy_service.cc:140] P 20730d75ad954a82b126148f8152a9f3: Received BeginTabletCopySession request for tablet b6ba8250f4ef43bfbf8f517e86a131a1 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:52758)
I20260809 15:51:32.036492 10942 tablet_copy_service.cc:161] P 20730d75ad954a82b126148f8152a9f3: Beginning new tablet copy session on tablet b6ba8250f4ef43bfbf8f517e86a131a1 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:52758: session id = c7aa762318cb4e97875333f10679a9f9-b6ba8250f4ef43bfbf8f517e86a131a1
I20260809 15:51:32.045006 10942 tablet_copy_source_session.cc:205] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:32.065919 11704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6ba8250f4ef43bfbf8f517e86a131a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:32.103885 11642 raft_consensus.cc:1203] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:32.123124 11704 tablet_copy_client.cc:791] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:32.123723 11704 tablet_copy_client.cc:655] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:32.127406 11704 tablet_copy_client.cc:531] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:32.147395 11704 tablet_bootstrap.cc:492] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:32.191289 11704 tablet_bootstrap.cc:492] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:32.192209 11704 tablet_bootstrap.cc:492] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:32.193015 11704 ts_tablet_manager.cc:1397] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.046s user 0.015s sys 0.013s
I20260809 15:51:32.195701 11704 raft_consensus.cc:348] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:32.196591 11704 raft_consensus.cc:729] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:32.199496 11642 raft_consensus.cc:1203] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:32.211393 11704 consensus_queue.cc:260] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:32.215294 11704 ts_tablet_manager.cc:1428] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.022s user 0.005s sys 0.000s
I20260809 15:51:32.216822 10942 tablet_copy_service.cc:342] P 20730d75ad954a82b126148f8152a9f3: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-b6ba8250f4ef43bfbf8f517e86a131a1 received from {username='slave'} at 127.0.0.1:52758
I20260809 15:51:32.217181 10942 tablet_copy_service.cc:433] P 20730d75ad954a82b126148f8152a9f3: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-b6ba8250f4ef43bfbf8f517e86a131a1 on tablet b6ba8250f4ef43bfbf8f517e86a131a1 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:32.394965 11642 raft_consensus.cc:1203] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:32.442811 11642 raft_consensus.cc:1203] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:32.493891 11520 raft_consensus.cc:1050] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:32.496707 11520 consensus_queue.cc:237] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:32.505242 11642 raft_consensus.cc:1261] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:32.505579 10931 raft_consensus.cc:1261] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:32.505264 10847 raft_consensus.cc:1261] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:32.509930 11680 consensus_queue.cc:1035] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:32.511000 11693 consensus_queue.cc:1035] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:32.512895 11693 consensus_queue.cc:1035] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:32.533957 10847 raft_consensus.cc:2932] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:32.531756 11723 raft_consensus.cc:2932] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:32.544962 11642 raft_consensus.cc:2932] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:32.553342 10737 catalog_manager.cc:5495] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:32.557823 10931 raft_consensus.cc:2932] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:32.619838 11012 consensus_queue.cc:237] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:32.631744 11642 raft_consensus.cc:1261] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:32.632558 10841 raft_consensus.cc:1261] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:32.633150 11520 consensus_queue.cc:1035] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:32.634143 11693 consensus_queue.cc:1035] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:32.645392 11520 raft_consensus.cc:2932] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:32.647836 10841 raft_consensus.cc:2932] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:32.652295 11642 raft_consensus.cc:2932] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:32.655217 10719 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet f866570ff0c5456ba186b9234cb692c8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:32.677548 10733 catalog_manager.cc:5495] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:32.703632 10913 tablet_service.cc:1514] Processing DeleteTablet for tablet f866570ff0c5456ba186b9234cb692c8 with delete_type TABLET_DATA_TOMBSTONED (TS 20730d75ad954a82b126148f8152a9f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54216
I20260809 15:51:32.728601 11731 tablet_replica.cc:330] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3: stopping tablet replica
I20260809 15:51:32.730389 11731 raft_consensus.cc:2229] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:32.731405 11731 raft_consensus.cc:2258] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:32.736542 11731 ts_tablet_manager.cc:1905] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:32.751072 11731 ts_tablet_manager.cc:1918] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:32.751554 11731 log.cc:1196] T f866570ff0c5456ba186b9234cb692c8 P 20730d75ad954a82b126148f8152a9f3: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-1-root/wals/f866570ff0c5456ba186b9234cb692c8
I20260809 15:51:32.753299 10722 catalog_manager.cc:4841] TS 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): tablet f866570ff0c5456ba186b9234cb692c8 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:32.759258 11680 raft_consensus.cc:1050] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:32.761469 11680 consensus_queue.cc:237] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:32.767154 11642 raft_consensus.cc:1261] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:32.768455 10927 raft_consensus.cc:1261] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:32.769588 10841 raft_consensus.cc:1261] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:32.770627 11680 consensus_queue.cc:1035] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:32.777588 11693 consensus_queue.cc:1035] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:32.779263 11723 consensus_queue.cc:1035] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.005s
I20260809 15:51:32.793215 11508 raft_consensus.cc:1050] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:32.795990 11508 consensus_queue.cc:237] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:32.822479 11693 raft_consensus.cc:2932] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:32.824196 10841 raft_consensus.cc:2932] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:32.831375 11642 raft_consensus.cc:1261] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:32.832907 11684 consensus_queue.cc:1035] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:32.833539 10924 raft_consensus.cc:1261] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:32.834774 11713 consensus_queue.cc:1035] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:32.841718 11641 raft_consensus.cc:2932] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:32.842123 10927 raft_consensus.cc:2932] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:32.847625 11010 raft_consensus.cc:1261] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:32.856710 10733 catalog_manager.cc:5495] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:32.860654 11687 raft_consensus.cc:1050] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:32.863356 11687 consensus_queue.cc:237] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:32.866266 11713 consensus_queue.cc:1035] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:32.904577 11011 raft_consensus.cc:1261] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:32.908010 11640 raft_consensus.cc:1261] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:32.904274 11010 consensus_queue.cc:237] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:32.908643 11687 consensus_queue.cc:1035] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260809 15:51:32.914479 10844 raft_consensus.cc:1261] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:32.918884 11687 consensus_queue.cc:1035] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:32.920123 11720 consensus_queue.cc:1035] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:32.933005 11640 raft_consensus.cc:1261] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:32.957185 10924 raft_consensus.cc:1261] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:32.958317 11723 consensus_queue.cc:1035] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:51:32.989734 11642 raft_consensus.cc:2932] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:33.012022 11684 raft_consensus.cc:2932] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:33.017480 11723 consensus_queue.cc:1035] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:51:33.023888 11012 raft_consensus.cc:2932] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:33.040904 11693 raft_consensus.cc:2932] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:33.060343 11640 raft_consensus.cc:2932] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:33.095682 10719 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 0b9fb9f88a6847af87fef710d1a2793b with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:33.112737 10736 catalog_manager.cc:5495] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:33.124794 10931 raft_consensus.cc:2932] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
W20260809 15:51:33.157231 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 66731.4us
I20260809 15:51:33.176802 10736 catalog_manager.cc:5495] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:33.188292 10924 raft_consensus.cc:2932] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:33.196714 10843 consensus_queue.cc:237] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:33.187335 11750 raft_consensus.cc:2932] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:33.210907 11011 raft_consensus.cc:2932] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:33.222883 11641 raft_consensus.cc:2932] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:33.224745 11640 raft_consensus.cc:1261] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:33.228348 10926 raft_consensus.cc:1261] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:33.228507 10828 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b9fb9f88a6847af87fef710d1a2793b with delete_type TABLET_DATA_TOMBSTONED (TS a53441872c0c416b8e2f26f52399f536 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45572
I20260809 15:51:33.247576 10733 catalog_manager.cc:5495] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:33.252357 11749 consensus_queue.cc:1035] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:51:33.253800 11746 consensus_queue.cc:1035] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:51:33.273038 10844 raft_consensus.cc:2932] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:33.326906 10931 consensus_queue.cc:237] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:33.337347 11641 raft_consensus.cc:1261] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:33.341208 11382 consensus_queue.cc:1035] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260809 15:51:33.347669 10840 raft_consensus.cc:1261] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:33.354943 11750 consensus_queue.cc:1035] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:33.366482 11751 raft_consensus.cc:2932] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:33.374279 11641 raft_consensus.cc:2932] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:33.376317 10840 raft_consensus.cc:2932] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:33.391398 11755 tablet_replica.cc:330] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536: stopping tablet replica
I20260809 15:51:33.393504 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet b6ba8250f4ef43bfbf8f517e86a131a1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:33.399739 11755 raft_consensus.cc:2229] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:33.405246 11746 raft_consensus.cc:2932] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:33.407660 10924 raft_consensus.cc:2932] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:33.412681 11755 raft_consensus.cc:2258] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:33.427989 11640 raft_consensus.cc:2932] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:33.444621 11755 ts_tablet_manager.cc:1905] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:33.448689 10734 catalog_manager.cc:5495] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:33.463744 10998 tablet_service.cc:1514] Processing DeleteTablet for tablet b6ba8250f4ef43bfbf8f517e86a131a1 with delete_type TABLET_DATA_TOMBSTONED (TS d74944fa9fa14ec78b0640ea709ff3aa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50526
I20260809 15:51:33.464885 11755 ts_tablet_manager.cc:1918] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:33.465308 11755 log.cc:1196] T 0b9fb9f88a6847af87fef710d1a2793b P a53441872c0c416b8e2f26f52399f536: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-0-root/wals/0b9fb9f88a6847af87fef710d1a2793b
I20260809 15:51:33.467453 10722 catalog_manager.cc:4841] TS a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): tablet 0b9fb9f88a6847af87fef710d1a2793b (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:33.474458 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet dd8bcd39f37d403f9d8d8f43d870a878 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:33.482117 10736 catalog_manager.cc:5495] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:33.503631 11006 tablet_service.cc:1514] Processing DeleteTablet for tablet dd8bcd39f37d403f9d8d8f43d870a878 with delete_type TABLET_DATA_TOMBSTONED (TS d74944fa9fa14ec78b0640ea709ff3aa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50526
I20260809 15:51:33.506317 11761 tablet_replica.cc:330] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa: stopping tablet replica
I20260809 15:51:33.508136 11761 raft_consensus.cc:2229] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:33.509102 11761 raft_consensus.cc:2258] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:33.520485 11761 ts_tablet_manager.cc:1905] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:33.545192 11761 ts_tablet_manager.cc:1918] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:33.545567 11761 log.cc:1196] T b6ba8250f4ef43bfbf8f517e86a131a1 P d74944fa9fa14ec78b0640ea709ff3aa: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-2-root/wals/b6ba8250f4ef43bfbf8f517e86a131a1
I20260809 15:51:33.547644 10719 catalog_manager.cc:4841] TS d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): tablet b6ba8250f4ef43bfbf8f517e86a131a1 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:33.547927 11761 tablet_replica.cc:330] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa: stopping tablet replica
I20260809 15:51:33.561106 11761 raft_consensus.cc:2229] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:33.561998 11761 raft_consensus.cc:2258] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:33.566321 11761 ts_tablet_manager.cc:1905] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:33.591881 11761 ts_tablet_manager.cc:1918] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:33.592394 11761 log.cc:1196] T dd8bcd39f37d403f9d8d8f43d870a878 P d74944fa9fa14ec78b0640ea709ff3aa: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-2-root/wals/dd8bcd39f37d403f9d8d8f43d870a878
I20260809 15:51:33.593811 10719 catalog_manager.cc:4841] TS d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): tablet dd8bcd39f37d403f9d8d8f43d870a878 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:34.563856 11018 consensus_queue.cc:237] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
W20260809 15:51:34.570865 10978 consensus_peers.cc:487] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e80d8e2921024e68a1b7c56fda99ee80. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:34.571401 10840 raft_consensus.cc:1261] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:34.572742 10926 raft_consensus.cc:1261] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:34.574251 11680 consensus_queue.cc:1035] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:34.576009 11680 consensus_queue.cc:1035] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:34.599023 11762 raft_consensus.cc:2932] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:34.645114 10926 raft_consensus.cc:2932] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:34.670718 10734 catalog_manager.cc:5495] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:34.665741 11011 consensus_queue.cc:237] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:34.700134 10844 raft_consensus.cc:2932] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
W20260809 15:51:34.744841 10978 consensus_peers.cc:487] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 241f07ed14b041cdb37e8ec792283d52. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:34.748553 10926 raft_consensus.cc:1261] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:34.751040 11680 consensus_queue.cc:1035] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:34.751503 10844 raft_consensus.cc:1261] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:34.753975 11723 consensus_queue.cc:1035] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:34.764649 11723 raft_consensus.cc:2932] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:34.768544 10844 raft_consensus.cc:2932] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:34.783439 10926 raft_consensus.cc:2932] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:34.790817 10734 catalog_manager.cc:5495] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:34.823745 10924 consensus_queue.cc:237] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:34.833302 10844 raft_consensus.cc:1261] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260809 15:51:34.833710 10893 consensus_peers.cc:487] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: bf6504ce1c0d4241bfb97eb553c8df35. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:34.834645 11011 raft_consensus.cc:1261] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260809 15:51:34.834751 11750 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:34.836177 11764 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:34.842978 11764 raft_consensus.cc:2932] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:34.844172 10840 raft_consensus.cc:2932] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:34.860067 10733 catalog_manager.cc:5495] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:34.879191 11011 raft_consensus.cc:2932] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:34.885746 10926 consensus_queue.cc:237] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
W20260809 15:51:34.901322 10893 consensus_peers.cc:487] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: c695a0d40c7e4b04a1f7a1113954a8e4. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:34.903167 10840 raft_consensus.cc:1261] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:34.904780 11764 consensus_queue.cc:1035] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:34.907184 11018 raft_consensus.cc:1261] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:34.910171 11751 consensus_queue.cc:1035] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:34.919476 11751 raft_consensus.cc:2932] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:34.921648 11018 raft_consensus.cc:2932] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:34.921814 10840 raft_consensus.cc:2932] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:34.939371 10733 catalog_manager.cc:5495] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:35.138830 11787 ts_tablet_manager.cc:927] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:35.140810 11787 tablet_copy_client.cc:316] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.3:39223
I20260809 15:51:35.142491 11028 tablet_copy_service.cc:140] P d74944fa9fa14ec78b0640ea709ff3aa: Received BeginTabletCopySession request for tablet 241f07ed14b041cdb37e8ec792283d52 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:39978)
I20260809 15:51:35.143052 11028 tablet_copy_service.cc:161] P d74944fa9fa14ec78b0640ea709ff3aa: Beginning new tablet copy session on tablet 241f07ed14b041cdb37e8ec792283d52 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:39978: session id = c7aa762318cb4e97875333f10679a9f9-241f07ed14b041cdb37e8ec792283d52
I20260809 15:51:35.152525 11789 ts_tablet_manager.cc:927] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:35.162225 11789 tablet_copy_client.cc:316] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.3:39223
I20260809 15:51:35.163601 11027 tablet_copy_service.cc:140] P d74944fa9fa14ec78b0640ea709ff3aa: Received BeginTabletCopySession request for tablet e80d8e2921024e68a1b7c56fda99ee80 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:39978)
I20260809 15:51:35.164131 11027 tablet_copy_service.cc:161] P d74944fa9fa14ec78b0640ea709ff3aa: Beginning new tablet copy session on tablet e80d8e2921024e68a1b7c56fda99ee80 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:39978: session id = c7aa762318cb4e97875333f10679a9f9-e80d8e2921024e68a1b7c56fda99ee80
I20260809 15:51:35.169178 11028 tablet_copy_source_session.cc:205] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:35.171548 11027 tablet_copy_source_session.cc:205] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:35.173050 11787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 241f07ed14b041cdb37e8ec792283d52. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:35.188163 11789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e80d8e2921024e68a1b7c56fda99ee80. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:35.200932 11787 tablet_copy_client.cc:791] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:35.201524 11787 tablet_copy_client.cc:655] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:35.205367 11787 tablet_copy_client.cc:531] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:35.221086 11787 tablet_bootstrap.cc:492] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:35.227553 11789 tablet_copy_client.cc:791] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:35.228286 11789 tablet_copy_client.cc:655] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:35.256243 11792 ts_tablet_manager.cc:927] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:35.274266 11792 tablet_copy_client.cc:316] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.2:32783
I20260809 15:51:35.276714 10942 tablet_copy_service.cc:140] P 20730d75ad954a82b126148f8152a9f3: Received BeginTabletCopySession request for tablet bf6504ce1c0d4241bfb97eb553c8df35 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:52758)
I20260809 15:51:35.277456 10942 tablet_copy_service.cc:161] P 20730d75ad954a82b126148f8152a9f3: Beginning new tablet copy session on tablet bf6504ce1c0d4241bfb97eb553c8df35 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:52758: session id = c7aa762318cb4e97875333f10679a9f9-bf6504ce1c0d4241bfb97eb553c8df35
I20260809 15:51:35.293149 11787 tablet_bootstrap.cc:492] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:35.294106 11787 tablet_bootstrap.cc:492] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:35.294785 11787 ts_tablet_manager.cc:1397] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.074s user 0.026s sys 0.005s
I20260809 15:51:35.307221 11789 tablet_copy_client.cc:531] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:35.313177 11787 raft_consensus.cc:348] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:35.318714 11787 raft_consensus.cc:729] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:35.319384 11787 consensus_queue.cc:260] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:35.313997 11789 tablet_bootstrap.cc:492] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:35.318351 10942 tablet_copy_source_session.cc:205] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:35.331687 11792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6504ce1c0d4241bfb97eb553c8df35. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:35.359835 11792 tablet_copy_client.cc:791] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:35.360364 11792 tablet_copy_client.cc:655] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:35.365116 11792 tablet_copy_client.cc:531] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:35.375909 11787 ts_tablet_manager.cc:1428] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.081s user 0.005s sys 0.005s
I20260809 15:51:35.377691 11027 tablet_copy_service.cc:342] P d74944fa9fa14ec78b0640ea709ff3aa: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-241f07ed14b041cdb37e8ec792283d52 received from {username='slave'} at 127.0.0.1:39978
I20260809 15:51:35.378130 11027 tablet_copy_service.cc:433] P d74944fa9fa14ec78b0640ea709ff3aa: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-241f07ed14b041cdb37e8ec792283d52 on tablet 241f07ed14b041cdb37e8ec792283d52 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:35.391695 11787 ts_tablet_manager.cc:927] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:35.395331 11787 tablet_copy_client.cc:316] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.2:32783
I20260809 15:51:35.398986 11792 tablet_bootstrap.cc:492] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:35.400573 10942 tablet_copy_service.cc:140] P 20730d75ad954a82b126148f8152a9f3: Received BeginTabletCopySession request for tablet c695a0d40c7e4b04a1f7a1113954a8e4 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:52758)
I20260809 15:51:35.401144 10942 tablet_copy_service.cc:161] P 20730d75ad954a82b126148f8152a9f3: Beginning new tablet copy session on tablet c695a0d40c7e4b04a1f7a1113954a8e4 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:52758: session id = c7aa762318cb4e97875333f10679a9f9-c695a0d40c7e4b04a1f7a1113954a8e4
I20260809 15:51:35.418648 10942 tablet_copy_source_session.cc:205] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:35.422140 11787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c695a0d40c7e4b04a1f7a1113954a8e4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:35.487823 11792 tablet_bootstrap.cc:492] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:35.488473 11789 tablet_bootstrap.cc:492] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:35.488844 11792 tablet_bootstrap.cc:492] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:35.489328 11789 tablet_bootstrap.cc:492] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:35.489614 11792 ts_tablet_manager.cc:1397] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.091s user 0.022s sys 0.015s
I20260809 15:51:35.489998 11789 ts_tablet_manager.cc:1397] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.176s user 0.030s sys 0.043s
I20260809 15:51:35.492846 11789 raft_consensus.cc:348] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:35.493743 11789 raft_consensus.cc:729] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:35.495223 11789 consensus_queue.cc:260] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:35.498163 11789 ts_tablet_manager.cc:1428] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260809 15:51:35.503206 11792 raft_consensus.cc:348] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:35.504148 11792 raft_consensus.cc:729] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:35.505155 11787 tablet_copy_client.cc:791] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:35.504832 11792 consensus_queue.cc:260] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:35.506119 11787 tablet_copy_client.cc:655] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:35.506266 11027 tablet_copy_service.cc:342] P d74944fa9fa14ec78b0640ea709ff3aa: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-e80d8e2921024e68a1b7c56fda99ee80 received from {username='slave'} at 127.0.0.1:39978
I20260809 15:51:35.506749 11027 tablet_copy_service.cc:433] P d74944fa9fa14ec78b0640ea709ff3aa: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-e80d8e2921024e68a1b7c56fda99ee80 on tablet e80d8e2921024e68a1b7c56fda99ee80 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:35.508695 11792 ts_tablet_manager.cc:1428] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.019s user 0.009s sys 0.000s
I20260809 15:51:35.510594 10942 tablet_copy_service.cc:342] P 20730d75ad954a82b126148f8152a9f3: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-bf6504ce1c0d4241bfb97eb553c8df35 received from {username='slave'} at 127.0.0.1:52758
I20260809 15:51:35.511024 10942 tablet_copy_service.cc:433] P 20730d75ad954a82b126148f8152a9f3: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-bf6504ce1c0d4241bfb97eb553c8df35 on tablet bf6504ce1c0d4241bfb97eb553c8df35 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:35.510614 11787 tablet_copy_client.cc:531] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:35.524657 11787 tablet_bootstrap.cc:492] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:35.571561 11787 tablet_bootstrap.cc:492] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:35.572448 11787 tablet_bootstrap.cc:492] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:35.573164 11787 ts_tablet_manager.cc:1397] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.049s user 0.007s sys 0.013s
I20260809 15:51:35.575942 11787 raft_consensus.cc:348] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:35.576876 11787 raft_consensus.cc:729] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:35.577584 11787 consensus_queue.cc:260] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:35.580427 11787 ts_tablet_manager.cc:1428] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.007s user 0.002s sys 0.003s
I20260809 15:51:35.582486 11640 raft_consensus.cc:1203] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:35.583771 10942 tablet_copy_service.cc:342] P 20730d75ad954a82b126148f8152a9f3: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-c695a0d40c7e4b04a1f7a1113954a8e4 received from {username='slave'} at 127.0.0.1:52758
I20260809 15:51:35.584185 10942 tablet_copy_service.cc:433] P 20730d75ad954a82b126148f8152a9f3: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-c695a0d40c7e4b04a1f7a1113954a8e4 on tablet c695a0d40c7e4b04a1f7a1113954a8e4 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:35.615309 11640 raft_consensus.cc:1203] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:35.843514 11640 raft_consensus.cc:1203] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20260809 15:51:35.879117 11640 raft_consensus.cc:1203] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:36.098018 11778 raft_consensus.cc:1050] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:36.114369 11762 raft_consensus.cc:1050] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:36.118237 11778 consensus_queue.cc:237] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:36.121415 11762 consensus_queue.cc:237] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:36.133540 11640 raft_consensus.cc:1261] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:36.138751 11723 consensus_queue.cc:1035] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:36.157667 10931 raft_consensus.cc:1261] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:36.165577 11723 consensus_queue.cc:1035] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:36.169833 11640 raft_consensus.cc:1261] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:36.172607 10924 raft_consensus.cc:1261] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:36.172627 11804 consensus_queue.cc:1035] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:36.175449 10843 raft_consensus.cc:1261] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:36.177284 11723 consensus_queue.cc:1035] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:36.178743 10844 raft_consensus.cc:1261] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:36.179847 11723 consensus_queue.cc:1035] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:36.188318 11723 consensus_queue.cc:1035] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:36.191890 11805 raft_consensus.cc:2932] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.204064 10924 raft_consensus.cc:2932] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.209496 11762 raft_consensus.cc:2932] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.220257 11640 raft_consensus.cc:2932] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.224668 10843 raft_consensus.cc:2932] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.236994 10734 catalog_manager.cc:5495] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.248106 10926 raft_consensus.cc:2932] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.257798 10848 raft_consensus.cc:2932] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.219816 11641 raft_consensus.cc:2932] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.284533 11799 raft_consensus.cc:1050] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:36.287043 11799 consensus_queue.cc:237] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:36.296348 11640 raft_consensus.cc:1261] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:36.298939 11018 raft_consensus.cc:1261] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:36.300587 11782 consensus_queue.cc:1035] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:36.302067 11751 consensus_queue.cc:1035] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:36.315646 10848 raft_consensus.cc:1261] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:36.325762 11822 consensus_queue.cc:1035] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:36.340942 11018 consensus_queue.cc:237] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:36.350515 11641 raft_consensus.cc:1261] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
W20260809 15:51:36.351143 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 49882.2us
I20260809 15:51:36.357568 10733 catalog_manager.cc:5495] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.360627 11762 consensus_queue.cc:1035] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:36.386726 11805 raft_consensus.cc:2932] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.391181 10926 raft_consensus.cc:1261] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:36.396744 11818 raft_consensus.cc:1050] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:36.397531 11640 raft_consensus.cc:2932] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.400470 11818 consensus_queue.cc:237] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:36.410902 11811 consensus_queue.cc:1035] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:36.413527 11011 raft_consensus.cc:2932] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.414928 10931 raft_consensus.cc:2932] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.417497 10719 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet e80d8e2921024e68a1b7c56fda99ee80 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:36.406173 11820 raft_consensus.cc:2932] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.420496 11010 consensus_queue.cc:237] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:36.428241 10844 raft_consensus.cc:1261] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:36.428999 11641 raft_consensus.cc:1261] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:36.429942 11805 consensus_queue.cc:1035] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:36.431186 11811 consensus_queue.cc:1035] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:36.431565 11641 raft_consensus.cc:2932] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.440196 11723 raft_consensus.cc:2932] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.445837 11642 raft_consensus.cc:2932] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.452993 10719 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 241f07ed14b041cdb37e8ec792283d52 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:36.453976 10844 raft_consensus.cc:2932] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.436035 10734 catalog_manager.cc:5495] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:36.479001 11011 raft_consensus.cc:1261] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:51:36.481786 11799 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:51:36.503808 11642 raft_consensus.cc:1261] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:51:36.504488 10843 raft_consensus.cc:2932] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.508469 11751 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:36.515563 10844 raft_consensus.cc:1261] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:51:36.523546 10736 catalog_manager.cc:5495] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:36.529027 11822 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:36.537806 10828 tablet_service.cc:1514] Processing DeleteTablet for tablet e80d8e2921024e68a1b7c56fda99ee80 with delete_type TABLET_DATA_TOMBSTONED (TS a53441872c0c416b8e2f26f52399f536 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45572
I20260809 15:51:36.548743 10737 catalog_manager.cc:5495] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.566218 10923 consensus_queue.cc:237] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:36.588725 11829 tablet_replica.cc:330] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536: stopping tablet replica
I20260809 15:51:36.589694 11829 raft_consensus.cc:2229] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:36.590602 11829 raft_consensus.cc:2258] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:36.596136 11641 raft_consensus.cc:1261] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:36.598006 11820 consensus_queue.cc:1035] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:51:36.600989 10913 tablet_service.cc:1514] Processing DeleteTablet for tablet 241f07ed14b041cdb37e8ec792283d52 with delete_type TABLET_DATA_TOMBSTONED (TS 20730d75ad954a82b126148f8152a9f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54216
I20260809 15:51:36.601625 11829 ts_tablet_manager.cc:1905] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:36.604653 11011 raft_consensus.cc:1261] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:36.610314 11818 consensus_queue.cc:1035] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:36.625938 11011 raft_consensus.cc:2932] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.624529 11820 raft_consensus.cc:2932] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.657104 10733 catalog_manager.cc:5495] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.660933 11751 raft_consensus.cc:2932] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.666147 11833 tablet_replica.cc:330] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3: stopping tablet replica
I20260809 15:51:36.672341 10828 tablet_service.cc:1514] Processing DeleteTablet for tablet c695a0d40c7e4b04a1f7a1113954a8e4 with delete_type TABLET_DATA_TOMBSTONED (TS a53441872c0c416b8e2f26f52399f536 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45572
I20260809 15:51:36.680348 11833 raft_consensus.cc:2229] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:36.683672 11833 raft_consensus.cc:2258] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:36.681936 11010 raft_consensus.cc:2932] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.679631 11642 raft_consensus.cc:2932] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.675078 11641 raft_consensus.cc:2932] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.700302 10733 catalog_manager.cc:5495] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:36.718259 11829 ts_tablet_manager.cc:1918] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:36.718860 11829 log.cc:1196] T e80d8e2921024e68a1b7c56fda99ee80 P a53441872c0c416b8e2f26f52399f536: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-0-root/wals/e80d8e2921024e68a1b7c56fda99ee80
I20260809 15:51:36.718907 11833 ts_tablet_manager.cc:1905] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:36.720695 10722 catalog_manager.cc:4841] TS a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): tablet e80d8e2921024e68a1b7c56fda99ee80 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:36.723198 11829 tablet_replica.cc:330] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536: stopping tablet replica
I20260809 15:51:36.731508 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet c695a0d40c7e4b04a1f7a1113954a8e4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260809 15:51:36.766592 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 71257.1us
I20260809 15:51:36.781211 10848 raft_consensus.cc:2932] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.788154 10931 consensus_queue.cc:237] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:36.804901 11833 ts_tablet_manager.cc:1918] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:36.805631 11833 log.cc:1196] T 241f07ed14b041cdb37e8ec792283d52 P 20730d75ad954a82b126148f8152a9f3: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-1-root/wals/241f07ed14b041cdb37e8ec792283d52
I20260809 15:51:36.808802 10722 catalog_manager.cc:4841] TS 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): tablet 241f07ed14b041cdb37e8ec792283d52 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:36.809171 11641 raft_consensus.cc:1261] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:36.810660 11820 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:51:36.818420 11829 raft_consensus.cc:2229] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:36.819535 11829 raft_consensus.cc:2258] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:36.823792 11829 ts_tablet_manager.cc:1905] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:36.825202 10848 raft_consensus.cc:1261] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:36.830780 11751 raft_consensus.cc:2932] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.832758 11641 raft_consensus.cc:2932] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.833811 11820 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:51:36.838805 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet bf6504ce1c0d4241bfb97eb553c8df35 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:36.840188 10848 raft_consensus.cc:2932] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:36.850103 10733 catalog_manager.cc:5495] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 4 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:36.880422 11829 ts_tablet_manager.cc:1918] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:36.880864 11829 log.cc:1196] T c695a0d40c7e4b04a1f7a1113954a8e4 P a53441872c0c416b8e2f26f52399f536: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-0-root/wals/c695a0d40c7e4b04a1f7a1113954a8e4
I20260809 15:51:36.882251 10722 catalog_manager.cc:4841] TS a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): tablet c695a0d40c7e4b04a1f7a1113954a8e4 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:36.887192 10998 tablet_service.cc:1514] Processing DeleteTablet for tablet bf6504ce1c0d4241bfb97eb553c8df35 with delete_type TABLET_DATA_TOMBSTONED (TS d74944fa9fa14ec78b0640ea709ff3aa not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:50526
I20260809 15:51:36.891775 11835 tablet_replica.cc:330] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa: stopping tablet replica
I20260809 15:51:36.892729 11835 raft_consensus.cc:2229] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:36.893254 11835 raft_consensus.cc:2258] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:36.897294 11835 ts_tablet_manager.cc:1905] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:36.916988 11835 ts_tablet_manager.cc:1918] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260809 15:51:36.917297 11835 log.cc:1196] T bf6504ce1c0d4241bfb97eb553c8df35 P d74944fa9fa14ec78b0640ea709ff3aa: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-2-root/wals/bf6504ce1c0d4241bfb97eb553c8df35
I20260809 15:51:36.918828 10719 catalog_manager.cc:4841] TS d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): tablet bf6504ce1c0d4241bfb97eb553c8df35 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:37.905910 11010 consensus_queue.cc:237] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:37.912247 10931 raft_consensus.cc:1261] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:37.912494 10843 raft_consensus.cc:1261] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 15:51:37.913332 10978 consensus_peers.cc:487] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1faddf7ecc6e48ba9306d7ece1f936b1. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:37.914219 11838 consensus_queue.cc:1035] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:37.915186 11811 consensus_queue.cc:1035] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:37.930832 11811 raft_consensus.cc:2932] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:37.933157 10843 raft_consensus.cc:2932] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:37.949771 10733 catalog_manager.cc:5495] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:37.956964 10923 raft_consensus.cc:2932] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:37.968614 11018 consensus_queue.cc:237] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:37.981837 10923 raft_consensus.cc:1261] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 15:51:37.982038 10978 consensus_peers.cc:487] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 34001ced69f34c6f930f5218eef46524. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:37.982205 10848 raft_consensus.cc:1261] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:37.983127 11811 consensus_queue.cc:1035] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:38.028766 11847 consensus_queue.cc:1035] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:38.095804 10923 raft_consensus.cc:2932] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:38.090312 11811 raft_consensus.cc:2932] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
W20260809 15:51:38.206670 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 54759.7us
I20260809 15:51:38.248339 10736 catalog_manager.cc:5495] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:38.261708 10923 consensus_queue.cc:237] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:38.274642 10848 raft_consensus.cc:1261] T 81af4312dd044dfa85b9067b2fe4974c P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 15:51:38.274652 10893 consensus_peers.cc:487] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 81af4312dd044dfa85b9067b2fe4974c. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:38.278018 10848 raft_consensus.cc:2932] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:38.284206 11017 raft_consensus.cc:1261] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:38.288877 11861 consensus_queue.cc:1035] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:38.291978 11855 consensus_queue.cc:1035] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:38.312664 11861 raft_consensus.cc:2932] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:38.318284 11017 raft_consensus.cc:2932] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:38.321388 10848 raft_consensus.cc:2932] T 81af4312dd044dfa85b9067b2fe4974c P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:38.328864 10923 consensus_queue.cc:237] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:38.339071 10734 catalog_manager.cc:5495] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:38.342968 11017 raft_consensus.cc:1261] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 15:51:38.343987 10893 consensus_peers.cc:487] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 049fe6a2ea4f4a15a98b9c6c5755173d. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:38.346673 10840 raft_consensus.cc:1261] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:38.348701 11818 consensus_queue.cc:1035] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:38.350503 11861 consensus_queue.cc:1035] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:38.368261 11820 raft_consensus.cc:2932] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:38.369675 10848 raft_consensus.cc:2932] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:38.373526 11017 raft_consensus.cc:2932] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:38.387630 10736 catalog_manager.cc:5495] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:38.485663 11870 ts_tablet_manager.cc:927] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:38.487929 11870 tablet_copy_client.cc:316] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.3:39223
I20260809 15:51:38.490263 11027 tablet_copy_service.cc:140] P d74944fa9fa14ec78b0640ea709ff3aa: Received BeginTabletCopySession request for tablet 1faddf7ecc6e48ba9306d7ece1f936b1 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:39978)
I20260809 15:51:38.490896 11027 tablet_copy_service.cc:161] P d74944fa9fa14ec78b0640ea709ff3aa: Beginning new tablet copy session on tablet 1faddf7ecc6e48ba9306d7ece1f936b1 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:39978: session id = c7aa762318cb4e97875333f10679a9f9-1faddf7ecc6e48ba9306d7ece1f936b1
I20260809 15:51:38.499352 11872 ts_tablet_manager.cc:927] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:38.499982 11027 tablet_copy_source_session.cc:205] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:38.502887 11872 tablet_copy_client.cc:316] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.3:39223
I20260809 15:51:38.503652 11870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1faddf7ecc6e48ba9306d7ece1f936b1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:38.504716 11027 tablet_copy_service.cc:140] P d74944fa9fa14ec78b0640ea709ff3aa: Received BeginTabletCopySession request for tablet 34001ced69f34c6f930f5218eef46524 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:39978)
I20260809 15:51:38.505162 11027 tablet_copy_service.cc:161] P d74944fa9fa14ec78b0640ea709ff3aa: Beginning new tablet copy session on tablet 34001ced69f34c6f930f5218eef46524 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:39978: session id = c7aa762318cb4e97875333f10679a9f9-34001ced69f34c6f930f5218eef46524
I20260809 15:51:38.512054 11027 tablet_copy_source_session.cc:205] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:38.520117 11870 tablet_copy_client.cc:791] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:38.520782 11870 tablet_copy_client.cc:655] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:38.529857 11872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 34001ced69f34c6f930f5218eef46524. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:38.540776 11870 tablet_copy_client.cc:531] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:38.545652 11872 tablet_copy_client.cc:791] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:38.546257 11872 tablet_copy_client.cc:655] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:38.550593 11872 tablet_copy_client.cc:531] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:38.551404 11870 tablet_bootstrap.cc:492] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:38.561239 11872 tablet_bootstrap.cc:492] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:38.591131 11870 tablet_bootstrap.cc:492] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:38.592084 11870 tablet_bootstrap.cc:492] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:38.592746 11870 ts_tablet_manager.cc:1397] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.042s user 0.014s sys 0.018s
I20260809 15:51:38.595914 11870 raft_consensus.cc:348] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:38.596774 11870 raft_consensus.cc:729] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:38.597419 11870 consensus_queue.cc:260] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:38.607620 11870 ts_tablet_manager.cc:1428] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
I20260809 15:51:38.609158 11027 tablet_copy_service.cc:342] P d74944fa9fa14ec78b0640ea709ff3aa: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-1faddf7ecc6e48ba9306d7ece1f936b1 received from {username='slave'} at 127.0.0.1:39978
I20260809 15:51:38.609583 11027 tablet_copy_service.cc:433] P d74944fa9fa14ec78b0640ea709ff3aa: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-1faddf7ecc6e48ba9306d7ece1f936b1 on tablet 1faddf7ecc6e48ba9306d7ece1f936b1 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:38.624372 11872 tablet_bootstrap.cc:492] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:38.625357 11872 tablet_bootstrap.cc:492] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:38.626116 11872 ts_tablet_manager.cc:1397] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.065s user 0.033s sys 0.025s
I20260809 15:51:38.628455 11872 raft_consensus.cc:348] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:38.629274 11872 raft_consensus.cc:729] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:38.630030 11872 consensus_queue.cc:260] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:38.632429 11872 ts_tablet_manager.cc:1428] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260809 15:51:38.634531 11027 tablet_copy_service.cc:342] P d74944fa9fa14ec78b0640ea709ff3aa: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-34001ced69f34c6f930f5218eef46524 received from {username='slave'} at 127.0.0.1:39978
I20260809 15:51:38.634924 11027 tablet_copy_service.cc:433] P d74944fa9fa14ec78b0640ea709ff3aa: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-34001ced69f34c6f930f5218eef46524 on tablet 34001ced69f34c6f930f5218eef46524 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:38.730831 11872 ts_tablet_manager.cc:927] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:38.732441 11872 tablet_copy_client.cc:316] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.2:32783
I20260809 15:51:38.734181 10942 tablet_copy_service.cc:140] P 20730d75ad954a82b126148f8152a9f3: Received BeginTabletCopySession request for tablet 049fe6a2ea4f4a15a98b9c6c5755173d from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:52758)
I20260809 15:51:38.734632 10942 tablet_copy_service.cc:161] P 20730d75ad954a82b126148f8152a9f3: Beginning new tablet copy session on tablet 049fe6a2ea4f4a15a98b9c6c5755173d from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:52758: session id = c7aa762318cb4e97875333f10679a9f9-049fe6a2ea4f4a15a98b9c6c5755173d
I20260809 15:51:38.741428 10942 tablet_copy_source_session.cc:205] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:38.744896 11872 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 049fe6a2ea4f4a15a98b9c6c5755173d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:38.760464 11872 tablet_copy_client.cc:791] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:38.761018 11872 tablet_copy_client.cc:655] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:38.765468 11872 tablet_copy_client.cc:531] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:38.774026 11872 tablet_bootstrap.cc:492] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:38.791451 11872 tablet_bootstrap.cc:492] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:38.792186 11870 ts_tablet_manager.cc:927] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:38.792318 11872 tablet_bootstrap.cc:492] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:38.802254 11872 ts_tablet_manager.cc:1397] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.028s user 0.019s sys 0.000s
I20260809 15:51:38.814481 11870 tablet_copy_client.cc:316] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.2:32783
I20260809 15:51:38.806120 11872 raft_consensus.cc:348] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:38.815338 11872 raft_consensus.cc:729] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:38.816047 10942 tablet_copy_service.cc:140] P 20730d75ad954a82b126148f8152a9f3: Received BeginTabletCopySession request for tablet 81af4312dd044dfa85b9067b2fe4974c from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:52758)
I20260809 15:51:38.815994 11872 consensus_queue.cc:260] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:38.816615 10942 tablet_copy_service.cc:161] P 20730d75ad954a82b126148f8152a9f3: Beginning new tablet copy session on tablet 81af4312dd044dfa85b9067b2fe4974c from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:52758: session id = c7aa762318cb4e97875333f10679a9f9-81af4312dd044dfa85b9067b2fe4974c
I20260809 15:51:38.817930 11872 ts_tablet_manager.cc:1428] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.015s user 0.005s sys 0.000s
I20260809 15:51:38.824075 10942 tablet_copy_source_session.cc:205] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:38.826049 10942 tablet_copy_service.cc:342] P 20730d75ad954a82b126148f8152a9f3: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-049fe6a2ea4f4a15a98b9c6c5755173d received from {username='slave'} at 127.0.0.1:52758
I20260809 15:51:38.826416 10942 tablet_copy_service.cc:433] P 20730d75ad954a82b126148f8152a9f3: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-049fe6a2ea4f4a15a98b9c6c5755173d on tablet 049fe6a2ea4f4a15a98b9c6c5755173d with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:38.831643 11870 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 81af4312dd044dfa85b9067b2fe4974c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:38.848917 11870 tablet_copy_client.cc:791] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:38.849485 11870 tablet_copy_client.cc:655] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:38.854401 11870 tablet_copy_client.cc:531] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:38.863063 11870 tablet_bootstrap.cc:492] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:38.880183 11870 tablet_bootstrap.cc:492] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:38.880913 11870 tablet_bootstrap.cc:492] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:38.881521 11870 ts_tablet_manager.cc:1397] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.008s
I20260809 15:51:38.883929 11870 raft_consensus.cc:348] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:38.884784 11870 raft_consensus.cc:729] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:38.885426 11870 consensus_queue.cc:260] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:38.887804 11870 ts_tablet_manager.cc:1428] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260809 15:51:38.889360 10942 tablet_copy_service.cc:342] P 20730d75ad954a82b126148f8152a9f3: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-81af4312dd044dfa85b9067b2fe4974c received from {username='slave'} at 127.0.0.1:52758
I20260809 15:51:38.889739 10942 tablet_copy_service.cc:433] P 20730d75ad954a82b126148f8152a9f3: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-81af4312dd044dfa85b9067b2fe4974c on tablet 81af4312dd044dfa85b9067b2fe4974c with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:39.083904 11642 raft_consensus.cc:1203] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:39.108170 11642 raft_consensus.cc:1203] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:39.240370 11642 raft_consensus.cc:1203] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:39.271445 11642 raft_consensus.cc:1203] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:39.623678 11723 raft_consensus.cc:1050] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:39.625552 11723 consensus_queue.cc:237] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:39.631907 10923 raft_consensus.cc:1261] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:39.632571 10844 raft_consensus.cc:1261] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:39.632851 11642 raft_consensus.cc:1261] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:39.635154 11881 consensus_queue.cc:1035] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:39.636482 11881 consensus_queue.cc:1035] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:39.649938 11881 consensus_queue.cc:1035] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:39.662993 11881 raft_consensus.cc:1050] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:39.666054 11881 consensus_queue.cc:237] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:39.683483 11884 raft_consensus.cc:2932] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.689065 10923 raft_consensus.cc:2932] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.694356 11642 raft_consensus.cc:2932] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.697983 11641 raft_consensus.cc:1261] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:39.699613 11862 consensus_queue.cc:1035] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:39.699939 10931 raft_consensus.cc:1261] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:39.701570 11723 consensus_queue.cc:1035] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:39.715420 10734 catalog_manager.cc:5495] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.726212 10844 raft_consensus.cc:1261] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:39.731205 11889 consensus_queue.cc:1035] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:39.732213 10844 raft_consensus.cc:2932] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.742910 11723 raft_consensus.cc:2932] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.751145 10840 raft_consensus.cc:2932] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.760113 11882 raft_consensus.cc:1050] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:39.765230 10931 raft_consensus.cc:2932] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.774823 11641 raft_consensus.cc:2932] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.775290 11882 consensus_queue.cc:237] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:39.794401 11017 raft_consensus.cc:1261] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:39.797705 11641 raft_consensus.cc:1261] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:39.800390 11820 consensus_queue.cc:1035] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:39.807199 11898 consensus_queue.cc:1035] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:39.825003 11017 consensus_queue.cc:237] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:39.824615 10733 catalog_manager.cc:5495] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.836823 11641 raft_consensus.cc:1261] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:39.837154 10840 raft_consensus.cc:1261] T 81af4312dd044dfa85b9067b2fe4974c P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:39.838677 11862 consensus_queue.cc:1035] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:39.848899 11884 raft_consensus.cc:2932] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.850848 10840 raft_consensus.cc:1261] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:39.851756 11898 consensus_queue.cc:1035] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:39.855476 11011 consensus_queue.cc:237] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:39.856922 11723 consensus_queue.cc:1035] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:39.863561 11642 raft_consensus.cc:1261] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:39.865329 11723 consensus_queue.cc:1035] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:51:39.864312 11641 raft_consensus.cc:2932] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.878393 11882 raft_consensus.cc:2932] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.886714 11018 raft_consensus.cc:2932] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.888350 11820 raft_consensus.cc:1050] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:39.889707 10848 raft_consensus.cc:1261] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:39.888809 11889 raft_consensus.cc:2932] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.891870 10844 raft_consensus.cc:2932] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.894034 11891 consensus_queue.cc:1035] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:39.899791 10736 catalog_manager.cc:5495] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.899948 11641 raft_consensus.cc:2932] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.904496 10848 raft_consensus.cc:2932] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.898002 11820 consensus_queue.cc:237] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:39.918416 10719 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 1faddf7ecc6e48ba9306d7ece1f936b1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:39.919404 10719 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 34001ced69f34c6f930f5218eef46524 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:39.922447 10840 raft_consensus.cc:2932] T 81af4312dd044dfa85b9067b2fe4974c P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.928721 11011 raft_consensus.cc:1261] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:39.930382 10913 tablet_service.cc:1514] Processing DeleteTablet for tablet 34001ced69f34c6f930f5218eef46524 with delete_type TABLET_DATA_TOMBSTONED (TS 20730d75ad954a82b126148f8152a9f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54216
I20260809 15:51:39.931459 11898 consensus_queue.cc:1035] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:39.939503 11641 raft_consensus.cc:1261] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:39.943298 11641 raft_consensus.cc:2932] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.946456 11898 consensus_queue.cc:1035] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:39.945194 10737 catalog_manager.cc:5495] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.974546 10912 tablet_service.cc:1514] Processing DeleteTablet for tablet 1faddf7ecc6e48ba9306d7ece1f936b1 with delete_type TABLET_DATA_TOMBSTONED (TS 20730d75ad954a82b126148f8152a9f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54216
I20260809 15:51:39.979828 10843 raft_consensus.cc:1261] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:39.977733 10734 catalog_manager.cc:5495] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:39.986722 11852 raft_consensus.cc:2932] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:39.998242 11011 raft_consensus.cc:2932] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:40.005147 11900 consensus_queue.cc:1035] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:40.015609 11906 tablet_replica.cc:330] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3: stopping tablet replica
I20260809 15:51:40.017071 11906 raft_consensus.cc:2229] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:40.019052 11641 raft_consensus.cc:2932] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:40.027566 11906 raft_consensus.cc:2258] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:40.032019 11906 ts_tablet_manager.cc:1905] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:40.033154 10734 catalog_manager.cc:5495] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:40.033937 10931 consensus_queue.cc:237] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:40.041385 11017 raft_consensus.cc:1261] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:40.042738 11641 raft_consensus.cc:1261] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:40.044121 11882 consensus_queue.cc:1035] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:40.045418 11820 consensus_queue.cc:1035] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:40.053201 11820 raft_consensus.cc:2932] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:40.053808 10844 raft_consensus.cc:2932] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:40.066015 11011 raft_consensus.cc:2932] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:40.075584 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 81af4312dd044dfa85b9067b2fe4974c with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:40.076010 11641 raft_consensus.cc:2932] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:40.080751 11906 ts_tablet_manager.cc:1918] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:40.081209 11906 log.cc:1196] T 34001ced69f34c6f930f5218eef46524 P 20730d75ad954a82b126148f8152a9f3: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-1-root/wals/34001ced69f34c6f930f5218eef46524
I20260809 15:51:40.082934 11906 tablet_replica.cc:330] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3: stopping tablet replica
I20260809 15:51:40.082945 10722 catalog_manager.cc:4841] TS 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): tablet 34001ced69f34c6f930f5218eef46524 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:40.084263 11906 raft_consensus.cc:2229] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:40.085085 11906 raft_consensus.cc:2258] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:40.089607 11906 ts_tablet_manager.cc:1905] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:40.094296 10734 catalog_manager.cc:5495] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:40.111070 10828 tablet_service.cc:1514] Processing DeleteTablet for tablet 81af4312dd044dfa85b9067b2fe4974c with delete_type TABLET_DATA_TOMBSTONED (TS a53441872c0c416b8e2f26f52399f536 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45572
I20260809 15:51:40.115254 11910 tablet_replica.cc:330] T 81af4312dd044dfa85b9067b2fe4974c P a53441872c0c416b8e2f26f52399f536: stopping tablet replica
I20260809 15:51:40.116381 11910 raft_consensus.cc:2229] T 81af4312dd044dfa85b9067b2fe4974c P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:40.117141 11910 raft_consensus.cc:2258] T 81af4312dd044dfa85b9067b2fe4974c P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:40.120880 11910 ts_tablet_manager.cc:1905] T 81af4312dd044dfa85b9067b2fe4974c P a53441872c0c416b8e2f26f52399f536: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:40.128536 11906 ts_tablet_manager.cc:1918] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:40.128985 11906 log.cc:1196] T 1faddf7ecc6e48ba9306d7ece1f936b1 P 20730d75ad954a82b126148f8152a9f3: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-1-root/wals/1faddf7ecc6e48ba9306d7ece1f936b1
I20260809 15:51:40.130712 10722 catalog_manager.cc:4841] TS 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): tablet 1faddf7ecc6e48ba9306d7ece1f936b1 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:40.140062 11910 ts_tablet_manager.cc:1918] T 81af4312dd044dfa85b9067b2fe4974c P a53441872c0c416b8e2f26f52399f536: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:40.140573 11910 log.cc:1196] T 81af4312dd044dfa85b9067b2fe4974c P a53441872c0c416b8e2f26f52399f536: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-0-root/wals/81af4312dd044dfa85b9067b2fe4974c
I20260809 15:51:40.142033 10722 catalog_manager.cc:4841] TS a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): tablet 81af4312dd044dfa85b9067b2fe4974c (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:40.504566 10931 consensus_queue.cc:237] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:40.508831 11641 raft_consensus.cc:1261] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:40.509595 10844 raft_consensus.cc:1261] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:40.509944 11900 consensus_queue.cc:1035] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:40.510883 11820 consensus_queue.cc:1035] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:40.517187 11852 raft_consensus.cc:2932] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:40.519549 11641 raft_consensus.cc:2932] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:40.526109 10844 raft_consensus.cc:2932] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:40.529469 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 049fe6a2ea4f4a15a98b9c6c5755173d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:40.532821 10733 catalog_manager.cc:5495] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:40.547513 10998 tablet_service.cc:1514] Processing DeleteTablet for tablet 049fe6a2ea4f4a15a98b9c6c5755173d with delete_type TABLET_DATA_TOMBSTONED (TS d74944fa9fa14ec78b0640ea709ff3aa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50526
I20260809 15:51:40.549309 11912 tablet_replica.cc:330] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa: stopping tablet replica
I20260809 15:51:40.560220 11912 raft_consensus.cc:2229] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:40.560961 11912 raft_consensus.cc:2258] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:40.563535 11912 ts_tablet_manager.cc:1905] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:40.575078 11912 ts_tablet_manager.cc:1918] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:40.575426 11912 log.cc:1196] T 049fe6a2ea4f4a15a98b9c6c5755173d P d74944fa9fa14ec78b0640ea709ff3aa: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-2-root/wals/049fe6a2ea4f4a15a98b9c6c5755173d
I20260809 15:51:40.577033 10719 catalog_manager.cc:4841] TS d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): tablet 049fe6a2ea4f4a15a98b9c6c5755173d (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:41.608112 11011 consensus_queue.cc:237] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
W20260809 15:51:41.614590 10978 consensus_peers.cc:487] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 70c84efc43004b7b8ef2f9b869250509. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:41.616990 10923 raft_consensus.cc:1261] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:41.617756 10843 raft_consensus.cc:1261] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:41.618538 11889 consensus_queue.cc:1035] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:41.619495 11884 consensus_queue.cc:1035] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:41.626796 11884 raft_consensus.cc:2932] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:41.628050 10931 raft_consensus.cc:2932] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:41.629629 10843 raft_consensus.cc:2932] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:41.640434 10844 consensus_queue.cc:237] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:41.647010 10734 catalog_manager.cc:5495] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
W20260809 15:51:41.665624 10811 consensus_peers.cc:487] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 13ef36e0f4c040a7963771a1dcb2a286. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:41.667382 10931 raft_consensus.cc:1261] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:41.667480 11011 raft_consensus.cc:1261] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:41.669309 11859 consensus_queue.cc:1035] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:41.671097 11859 consensus_queue.cc:1035] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:41.681535 11904 raft_consensus.cc:2932] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:41.685694 11017 raft_consensus.cc:2932] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:41.704056 10923 consensus_queue.cc:237] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:41.712298 10931 raft_consensus.cc:2932] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
W20260809 15:51:41.718900 10893 consensus_peers.cc:487] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6a89bcd329414d49a3bfdf5ac68523e9. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:41.719293 10844 raft_consensus.cc:1261] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:41.720633 11900 consensus_queue.cc:1035] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:41.721021 10736 catalog_manager.cc:5495] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:41.732049 11017 raft_consensus.cc:1261] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:41.731315 11913 raft_consensus.cc:2932] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:41.734704 11820 consensus_queue.cc:1035] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:41.736522 10843 raft_consensus.cc:2932] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:41.744416 11011 consensus_queue.cc:237] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:41.762486 10923 raft_consensus.cc:1261] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:41.764279 10844 raft_consensus.cc:1261] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:41.764362 11918 consensus_queue.cc:1035] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:41.768296 11017 raft_consensus.cc:2932] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
W20260809 15:51:41.774930 10978 consensus_peers.cc:487] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 73ac44823c384ed99817c1ba3bba75ea. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:41.762889 10734 catalog_manager.cc:5495] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:41.780285 11884 consensus_queue.cc:1035] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:41.809052 10844 raft_consensus.cc:2932] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:41.809958 11889 raft_consensus.cc:2932] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:41.817377 10923 raft_consensus.cc:2932] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:41.824443 10733 catalog_manager.cc:5495] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:42.010474 11939 ts_tablet_manager.cc:927] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:42.012708 11939 tablet_copy_client.cc:316] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.3:39223
I20260809 15:51:42.014038 11027 tablet_copy_service.cc:140] P d74944fa9fa14ec78b0640ea709ff3aa: Received BeginTabletCopySession request for tablet 70c84efc43004b7b8ef2f9b869250509 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:39978)
I20260809 15:51:42.014546 11027 tablet_copy_service.cc:161] P d74944fa9fa14ec78b0640ea709ff3aa: Beginning new tablet copy session on tablet 70c84efc43004b7b8ef2f9b869250509 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:39978: session id = c7aa762318cb4e97875333f10679a9f9-70c84efc43004b7b8ef2f9b869250509
I20260809 15:51:42.020975 11027 tablet_copy_source_session.cc:205] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:42.023844 11939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 70c84efc43004b7b8ef2f9b869250509. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:42.037178 11939 tablet_copy_client.cc:791] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:42.037874 11939 tablet_copy_client.cc:655] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:42.041182 11939 tablet_copy_client.cc:531] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:42.048714 11939 tablet_bootstrap.cc:492] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:42.065809 11939 tablet_bootstrap.cc:492] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:42.066601 11939 tablet_bootstrap.cc:492] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:42.067188 11939 ts_tablet_manager.cc:1397] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.019s user 0.020s sys 0.000s
I20260809 15:51:42.069221 11939 raft_consensus.cc:348] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:42.069998 11939 raft_consensus.cc:729] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:42.070528 11939 consensus_queue.cc:260] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:42.073789 11939 ts_tablet_manager.cc:1428] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260809 15:51:42.078836 11027 tablet_copy_service.cc:342] P d74944fa9fa14ec78b0640ea709ff3aa: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-70c84efc43004b7b8ef2f9b869250509 received from {username='slave'} at 127.0.0.1:39978
I20260809 15:51:42.079274 11027 tablet_copy_service.cc:433] P d74944fa9fa14ec78b0640ea709ff3aa: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-70c84efc43004b7b8ef2f9b869250509 on tablet 70c84efc43004b7b8ef2f9b869250509 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:42.087680 11939 ts_tablet_manager.cc:927] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615)
I20260809 15:51:42.089207 11939 tablet_copy_client.cc:316] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.1:46615
I20260809 15:51:42.091120 10858 tablet_copy_service.cc:140] P a53441872c0c416b8e2f26f52399f536: Received BeginTabletCopySession request for tablet 13ef36e0f4c040a7963771a1dcb2a286 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:54342)
I20260809 15:51:42.091662 10858 tablet_copy_service.cc:161] P a53441872c0c416b8e2f26f52399f536: Beginning new tablet copy session on tablet 13ef36e0f4c040a7963771a1dcb2a286 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:54342: session id = c7aa762318cb4e97875333f10679a9f9-13ef36e0f4c040a7963771a1dcb2a286
I20260809 15:51:42.100256 10858 tablet_copy_source_session.cc:205] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:42.103112 11939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13ef36e0f4c040a7963771a1dcb2a286. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:42.117436 11939 tablet_copy_client.cc:791] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:42.118026 11939 tablet_copy_client.cc:655] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:42.121480 11939 tablet_copy_client.cc:531] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:42.130291 11939 tablet_bootstrap.cc:492] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:42.166674 11939 tablet_bootstrap.cc:492] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:42.167738 11939 tablet_bootstrap.cc:492] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:42.168700 11939 ts_tablet_manager.cc:1397] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.039s user 0.029s sys 0.008s
I20260809 15:51:42.174798 11939 raft_consensus.cc:348] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:42.176199 11939 raft_consensus.cc:729] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:42.177166 11939 consensus_queue.cc:260] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:42.180474 11939 ts_tablet_manager.cc:1428] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.010s user 0.003s sys 0.004s
I20260809 15:51:42.183020 10858 tablet_copy_service.cc:342] P a53441872c0c416b8e2f26f52399f536: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-13ef36e0f4c040a7963771a1dcb2a286 received from {username='slave'} at 127.0.0.1:54342
I20260809 15:51:42.183465 10858 tablet_copy_service.cc:433] P a53441872c0c416b8e2f26f52399f536: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-13ef36e0f4c040a7963771a1dcb2a286 on tablet 13ef36e0f4c040a7963771a1dcb2a286 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:42.230805 11939 ts_tablet_manager.cc:927] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:42.233184 11939 tablet_copy_client.cc:316] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.3:39223
I20260809 15:51:42.235631 11027 tablet_copy_service.cc:140] P d74944fa9fa14ec78b0640ea709ff3aa: Received BeginTabletCopySession request for tablet 73ac44823c384ed99817c1ba3bba75ea from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:39978)
I20260809 15:51:42.236188 11027 tablet_copy_service.cc:161] P d74944fa9fa14ec78b0640ea709ff3aa: Beginning new tablet copy session on tablet 73ac44823c384ed99817c1ba3bba75ea from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:39978: session id = c7aa762318cb4e97875333f10679a9f9-73ac44823c384ed99817c1ba3bba75ea
I20260809 15:51:42.239096 11946 ts_tablet_manager.cc:927] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:42.246078 11027 tablet_copy_source_session.cc:205] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:42.249203 11946 tablet_copy_client.cc:316] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.2:32783
I20260809 15:51:42.249723 11939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73ac44823c384ed99817c1ba3bba75ea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:42.250703 10942 tablet_copy_service.cc:140] P 20730d75ad954a82b126148f8152a9f3: Received BeginTabletCopySession request for tablet 6a89bcd329414d49a3bfdf5ac68523e9 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:52758)
I20260809 15:51:42.251158 10942 tablet_copy_service.cc:161] P 20730d75ad954a82b126148f8152a9f3: Beginning new tablet copy session on tablet 6a89bcd329414d49a3bfdf5ac68523e9 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:52758: session id = c7aa762318cb4e97875333f10679a9f9-6a89bcd329414d49a3bfdf5ac68523e9
I20260809 15:51:42.256805 10942 tablet_copy_source_session.cc:205] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:42.264101 11939 tablet_copy_client.cc:791] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:42.284233 11939 tablet_copy_client.cc:655] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:42.285254 11946 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a89bcd329414d49a3bfdf5ac68523e9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:42.296905 11939 tablet_copy_client.cc:531] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:42.309075 11939 tablet_bootstrap.cc:492] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:42.314136 11946 tablet_copy_client.cc:791] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:42.314747 11946 tablet_copy_client.cc:655] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:42.318136 11946 tablet_copy_client.cc:531] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:42.326427 11946 tablet_bootstrap.cc:492] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:42.326576 11939 tablet_bootstrap.cc:492] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:42.327448 11939 tablet_bootstrap.cc:492] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:42.328101 11939 ts_tablet_manager.cc:1397] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.019s user 0.010s sys 0.008s
I20260809 15:51:42.332511 11939 raft_consensus.cc:348] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:42.333323 11939 raft_consensus.cc:729] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:42.333982 11939 consensus_queue.cc:260] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:42.336169 11939 ts_tablet_manager.cc:1428] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260809 15:51:42.338371 11027 tablet_copy_service.cc:342] P d74944fa9fa14ec78b0640ea709ff3aa: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-73ac44823c384ed99817c1ba3bba75ea received from {username='slave'} at 127.0.0.1:39978
I20260809 15:51:42.338735 11027 tablet_copy_service.cc:433] P d74944fa9fa14ec78b0640ea709ff3aa: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-73ac44823c384ed99817c1ba3bba75ea on tablet 73ac44823c384ed99817c1ba3bba75ea with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:42.350728 11946 tablet_bootstrap.cc:492] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:42.351646 11946 tablet_bootstrap.cc:492] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:42.353156 11946 ts_tablet_manager.cc:1397] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.027s user 0.012s sys 0.012s
I20260809 15:51:42.355768 11946 raft_consensus.cc:348] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:42.356534 11946 raft_consensus.cc:729] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:42.357093 11946 consensus_queue.cc:260] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:42.359514 11946 ts_tablet_manager.cc:1428] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260809 15:51:42.361020 10942 tablet_copy_service.cc:342] P 20730d75ad954a82b126148f8152a9f3: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-6a89bcd329414d49a3bfdf5ac68523e9 received from {username='slave'} at 127.0.0.1:52758
I20260809 15:51:42.361362 10942 tablet_copy_service.cc:433] P 20730d75ad954a82b126148f8152a9f3: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-6a89bcd329414d49a3bfdf5ac68523e9 on tablet 6a89bcd329414d49a3bfdf5ac68523e9 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:42.573092 11641 raft_consensus.cc:1203] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:42.689894 11641 raft_consensus.cc:1203] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:42.760707 11641 raft_consensus.cc:1203] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:42.777865 11641 raft_consensus.cc:1203] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:42.996394 11918 raft_consensus.cc:1050] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:42.998340 11918 consensus_queue.cc:237] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:43.004328 11641 raft_consensus.cc:1261] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:43.004468 10924 raft_consensus.cc:1261] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:43.004670 10840 raft_consensus.cc:1261] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:43.006481 11918 consensus_queue.cc:1035] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:43.007426 11889 consensus_queue.cc:1035] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:43.010686 11950 consensus_queue.cc:1035] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:43.033793 11954 raft_consensus.cc:2932] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.035065 10843 raft_consensus.cc:2932] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.047977 11642 raft_consensus.cc:2932] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.051594 10924 raft_consensus.cc:2932] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.069815 10733 catalog_manager.cc:5495] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.144590 11017 consensus_queue.cc:237] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:43.160930 11641 raft_consensus.cc:1261] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:43.162317 10923 raft_consensus.cc:1261] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:43.162171 11918 consensus_queue.cc:1035] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:43.163539 11950 consensus_queue.cc:1035] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:43.173395 10924 raft_consensus.cc:2932] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.171685 11954 raft_consensus.cc:2932] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.188200 11951 raft_consensus.cc:1050] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:43.188316 11641 raft_consensus.cc:2932] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.190498 11951 consensus_queue.cc:237] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:43.198658 10719 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 70c84efc43004b7b8ef2f9b869250509 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:43.206631 10734 catalog_manager.cc:5495] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.211103 11642 raft_consensus.cc:1261] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:43.211900 10924 raft_consensus.cc:1261] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:43.214455 11017 raft_consensus.cc:1261] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:43.215265 11921 consensus_queue.cc:1035] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:43.217073 11921 consensus_queue.cc:1035] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:43.217839 11951 consensus_queue.cc:1035] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:43.220248 11934 raft_consensus.cc:1050] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:43.225986 11934 consensus_queue.cc:237] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:43.228852 11918 raft_consensus.cc:1050] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:43.232650 11918 consensus_queue.cc:237] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:43.238904 10828 tablet_service.cc:1514] Processing DeleteTablet for tablet 70c84efc43004b7b8ef2f9b869250509 with delete_type TABLET_DATA_TOMBSTONED (TS a53441872c0c416b8e2f26f52399f536 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45572
I20260809 15:51:43.240031 11962 raft_consensus.cc:2932] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.241408 11012 raft_consensus.cc:2932] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.255663 11972 tablet_replica.cc:330] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536: stopping tablet replica
I20260809 15:51:43.259140 11972 raft_consensus.cc:2229] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:43.259935 11642 raft_consensus.cc:2932] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.260980 11641 raft_consensus.cc:1261] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:43.259891 11972 raft_consensus.cc:2258] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:43.261965 11011 raft_consensus.cc:1261] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:43.262156 11952 consensus_queue.cc:1035] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:43.265784 11820 consensus_queue.cc:1035] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:43.268008 11972 ts_tablet_manager.cc:1905] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:43.271773 10848 raft_consensus.cc:1261] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:43.273298 11820 consensus_queue.cc:1035] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:43.280231 10923 raft_consensus.cc:2932] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.286623 10736 catalog_manager.cc:5495] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:43.299520 11972 ts_tablet_manager.cc:1918] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:43.299968 11972 log.cc:1196] T 70c84efc43004b7b8ef2f9b869250509 P a53441872c0c416b8e2f26f52399f536: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-0-root/wals/70c84efc43004b7b8ef2f9b869250509
I20260809 15:51:43.301911 11642 raft_consensus.cc:1261] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:43.306507 10840 raft_consensus.cc:1261] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:43.307281 10722 catalog_manager.cc:4841] TS a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): tablet 70c84efc43004b7b8ef2f9b869250509 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:43.312860 11950 consensus_queue.cc:1035] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:43.315402 11918 consensus_queue.cc:1035] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:43.309528 10843 consensus_queue.cc:237] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:43.330768 11639 raft_consensus.cc:1261] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:43.333338 11921 consensus_queue.cc:1035] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:43.334432 10924 raft_consensus.cc:1261] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:43.334842 11011 raft_consensus.cc:1261] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:43.339193 11962 consensus_queue.cc:1035] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:43.343286 11639 raft_consensus.cc:2932] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.341917 11962 raft_consensus.cc:2932] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.359428 11934 raft_consensus.cc:2932] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.374071 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 13ef36e0f4c040a7963771a1dcb2a286 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:43.373857 11011 raft_consensus.cc:2932] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.382349 11950 consensus_queue.cc:1035] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:43.383982 10736 catalog_manager.cc:5495] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:43.396451 10913 tablet_service.cc:1514] Processing DeleteTablet for tablet 13ef36e0f4c040a7963771a1dcb2a286 with delete_type TABLET_DATA_TOMBSTONED (TS 20730d75ad954a82b126148f8152a9f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54216
I20260809 15:51:43.371920 11641 raft_consensus.cc:2932] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.402959 11012 raft_consensus.cc:2932] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.401825 10737 catalog_manager.cc:5495] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:43.412416 11918 raft_consensus.cc:2932] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.440838 11979 tablet_replica.cc:330] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3: stopping tablet replica
I20260809 15:51:43.443009 10848 raft_consensus.cc:2932] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.448202 11979 raft_consensus.cc:2229] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:43.448865 11979 raft_consensus.cc:2258] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:43.464340 11979 ts_tablet_manager.cc:1905] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:43.481688 10923 raft_consensus.cc:2932] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.490995 11641 raft_consensus.cc:2932] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.491706 10924 consensus_queue.cc:237] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:43.498556 10843 raft_consensus.cc:2932] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.504979 11639 raft_consensus.cc:1261] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:43.505815 10848 raft_consensus.cc:1261] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:43.506263 11934 consensus_queue.cc:1035] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:43.512840 11952 consensus_queue.cc:1035] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:43.516667 11820 raft_consensus.cc:2932] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.519155 11979 ts_tablet_manager.cc:1918] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:43.528666 11979 log.cc:1196] T 13ef36e0f4c040a7963771a1dcb2a286 P 20730d75ad954a82b126148f8152a9f3: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-1-root/wals/13ef36e0f4c040a7963771a1dcb2a286
I20260809 15:51:43.523178 11639 raft_consensus.cc:2932] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.530530 10722 catalog_manager.cc:4841] TS 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): tablet 13ef36e0f4c040a7963771a1dcb2a286 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:43.532613 10848 raft_consensus.cc:2932] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.521839 10737 catalog_manager.cc:5495] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.535188 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 6a89bcd329414d49a3bfdf5ac68523e9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:43.553125 11011 consensus_queue.cc:237] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:43.556208 10736 catalog_manager.cc:5495] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.560346 11639 raft_consensus.cc:1261] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:43.561861 10924 raft_consensus.cc:1261] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:43.562647 11950 consensus_queue.cc:1035] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:43.563611 11980 consensus_queue.cc:1035] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:51:43.567328 10998 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a89bcd329414d49a3bfdf5ac68523e9 with delete_type TABLET_DATA_TOMBSTONED (TS d74944fa9fa14ec78b0640ea709ff3aa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50526
I20260809 15:51:43.570355 11981 tablet_replica.cc:330] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa: stopping tablet replica
I20260809 15:51:43.571511 11981 raft_consensus.cc:2229] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:43.572193 11981 raft_consensus.cc:2258] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:43.572197 11980 raft_consensus.cc:2932] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.575454 11981 ts_tablet_manager.cc:1905] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:43.574568 11639 raft_consensus.cc:2932] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.587877 10719 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 73ac44823c384ed99817c1ba3bba75ea with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:43.588600 10737 catalog_manager.cc:5495] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.589483 10924 raft_consensus.cc:2932] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:43.597730 10828 tablet_service.cc:1514] Processing DeleteTablet for tablet 73ac44823c384ed99817c1ba3bba75ea with delete_type TABLET_DATA_TOMBSTONED (TS a53441872c0c416b8e2f26f52399f536 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45572
I20260809 15:51:43.598667 11972 tablet_replica.cc:330] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536: stopping tablet replica
I20260809 15:51:43.599624 11972 raft_consensus.cc:2229] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:43.600229 11972 raft_consensus.cc:2258] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:43.606587 11981 ts_tablet_manager.cc:1918] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:43.607057 11981 log.cc:1196] T 6a89bcd329414d49a3bfdf5ac68523e9 P d74944fa9fa14ec78b0640ea709ff3aa: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-2-root/wals/6a89bcd329414d49a3bfdf5ac68523e9
I20260809 15:51:43.609402 10719 catalog_manager.cc:4841] TS d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): tablet 6a89bcd329414d49a3bfdf5ac68523e9 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:43.610140 11972 ts_tablet_manager.cc:1905] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:43.623617 11972 ts_tablet_manager.cc:1918] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:43.623871 11972 log.cc:1196] T 73ac44823c384ed99817c1ba3bba75ea P a53441872c0c416b8e2f26f52399f536: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-0-root/wals/73ac44823c384ed99817c1ba3bba75ea
I20260809 15:51:43.625126 10722 catalog_manager.cc:4841] TS a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): tablet 73ac44823c384ed99817c1ba3bba75ea (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:44.671010 11012 consensus_queue.cc:237] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:44.679049 10923 raft_consensus.cc:1261] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:44.680114 10843 raft_consensus.cc:1261] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:44.680362 11980 consensus_queue.cc:1035] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260809 15:51:44.682720 10978 consensus_peers.cc:487] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2b4306bfa5464b0dbf8301756455ba50. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:44.683682 11980 consensus_queue.cc:1035] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:44.689978 11950 raft_consensus.cc:2932] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:44.693352 10923 raft_consensus.cc:2932] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:44.694406 10843 raft_consensus.cc:2932] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:44.711350 10924 consensus_queue.cc:237] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:44.718441 10736 catalog_manager.cc:5495] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:44.740248 11011 raft_consensus.cc:1261] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:44.741540 11934 consensus_queue.cc:1035] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:44.747717 10843 raft_consensus.cc:1261] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 15:51:44.749282 10893 consensus_peers.cc:487] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e42f46986036495ab1735d596ab06d6d. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:44.753713 11820 consensus_queue.cc:1035] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:44.764755 11991 raft_consensus.cc:2932] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:44.768610 11011 raft_consensus.cc:2932] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:44.772651 10843 raft_consensus.cc:2932] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:44.793244 10924 consensus_queue.cc:237] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:44.799782 10733 catalog_manager.cc:5495] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
W20260809 15:51:44.809751 10893 consensus_peers.cc:487] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 9c38f65ac0de428ba1d25c6f0bb6dc91. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:44.810509 11017 raft_consensus.cc:1261] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:44.811705 11820 consensus_queue.cc:1035] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:44.812885 10843 raft_consensus.cc:1261] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:44.814209 11820 consensus_queue.cc:1035] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:44.825798 11820 raft_consensus.cc:2932] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:44.827293 10843 raft_consensus.cc:2932] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:44.829331 11017 raft_consensus.cc:2932] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:44.841481 10733 catalog_manager.cc:5495] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:44.847672 10924 consensus_queue.cc:237] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
W20260809 15:51:44.856822 10893 consensus_peers.cc:487] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 11cab658217a44bebba11963c723e3dc. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:44.858840 10843 raft_consensus.cc:1261] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:44.859831 11991 consensus_queue.cc:1035] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:44.862712 11017 raft_consensus.cc:1261] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:44.864909 11991 consensus_queue.cc:1035] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:44.868849 10843 raft_consensus.cc:2932] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:44.871891 11820 raft_consensus.cc:2932] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:44.879410 11011 raft_consensus.cc:2932] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:44.886552 10733 catalog_manager.cc:5495] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:45.185729 12005 ts_tablet_manager.cc:927] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:45.187565 12005 tablet_copy_client.cc:316] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.3:39223
I20260809 15:51:45.189195 11027 tablet_copy_service.cc:140] P d74944fa9fa14ec78b0640ea709ff3aa: Received BeginTabletCopySession request for tablet 2b4306bfa5464b0dbf8301756455ba50 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:39978)
I20260809 15:51:45.189846 11027 tablet_copy_service.cc:161] P d74944fa9fa14ec78b0640ea709ff3aa: Beginning new tablet copy session on tablet 2b4306bfa5464b0dbf8301756455ba50 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:39978: session id = c7aa762318cb4e97875333f10679a9f9-2b4306bfa5464b0dbf8301756455ba50
I20260809 15:51:45.198951 11027 tablet_copy_source_session.cc:205] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:45.201202 12007 ts_tablet_manager.cc:927] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:45.202306 12005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b4306bfa5464b0dbf8301756455ba50. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:45.203281 12007 tablet_copy_client.cc:316] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.2:32783
I20260809 15:51:45.204945 10942 tablet_copy_service.cc:140] P 20730d75ad954a82b126148f8152a9f3: Received BeginTabletCopySession request for tablet e42f46986036495ab1735d596ab06d6d from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:52758)
I20260809 15:51:45.205413 10942 tablet_copy_service.cc:161] P 20730d75ad954a82b126148f8152a9f3: Beginning new tablet copy session on tablet e42f46986036495ab1735d596ab06d6d from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:52758: session id = c7aa762318cb4e97875333f10679a9f9-e42f46986036495ab1735d596ab06d6d
I20260809 15:51:45.212658 10942 tablet_copy_source_session.cc:205] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:45.215373 12007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e42f46986036495ab1735d596ab06d6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:45.219061 12005 tablet_copy_client.cc:791] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:45.219647 12005 tablet_copy_client.cc:655] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:45.223119 12005 tablet_copy_client.cc:531] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:45.231114 12005 tablet_bootstrap.cc:492] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:45.231599 12007 tablet_copy_client.cc:791] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:45.232136 12007 tablet_copy_client.cc:655] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:45.235234 12007 tablet_copy_client.cc:531] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:45.243968 12007 tablet_bootstrap.cc:492] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:45.274878 12005 tablet_bootstrap.cc:492] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:45.275851 12005 tablet_bootstrap.cc:492] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:45.276610 12005 ts_tablet_manager.cc:1397] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.046s user 0.021s sys 0.004s
I20260809 15:51:45.279323 12005 raft_consensus.cc:348] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:45.280288 12005 raft_consensus.cc:729] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:45.281375 12010 ts_tablet_manager.cc:927] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:45.281005 12005 consensus_queue.cc:260] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:45.285085 12005 ts_tablet_manager.cc:1428] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260809 15:51:45.289685 11027 tablet_copy_service.cc:342] P d74944fa9fa14ec78b0640ea709ff3aa: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-2b4306bfa5464b0dbf8301756455ba50 received from {username='slave'} at 127.0.0.1:39978
I20260809 15:51:45.290323 11027 tablet_copy_service.cc:433] P d74944fa9fa14ec78b0640ea709ff3aa: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-2b4306bfa5464b0dbf8301756455ba50 on tablet 2b4306bfa5464b0dbf8301756455ba50 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:45.293247 12010 tablet_copy_client.cc:316] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.2:32783
I20260809 15:51:45.294840 10942 tablet_copy_service.cc:140] P 20730d75ad954a82b126148f8152a9f3: Received BeginTabletCopySession request for tablet 11cab658217a44bebba11963c723e3dc from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:52758)
I20260809 15:51:45.295284 12007 tablet_bootstrap.cc:492] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:45.295336 10942 tablet_copy_service.cc:161] P 20730d75ad954a82b126148f8152a9f3: Beginning new tablet copy session on tablet 11cab658217a44bebba11963c723e3dc from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:52758: session id = c7aa762318cb4e97875333f10679a9f9-11cab658217a44bebba11963c723e3dc
I20260809 15:51:45.296147 12007 tablet_bootstrap.cc:492] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:45.296800 12007 ts_tablet_manager.cc:1397] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.053s user 0.026s sys 0.007s
I20260809 15:51:45.300702 12007 raft_consensus.cc:348] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:45.301520 12007 raft_consensus.cc:729] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:45.302193 12007 consensus_queue.cc:260] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:45.304297 10942 tablet_copy_source_session.cc:205] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:45.306545 12007 ts_tablet_manager.cc:1428] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.009s user 0.007s sys 0.001s
I20260809 15:51:45.307993 12010 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 11cab658217a44bebba11963c723e3dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:45.311645 10942 tablet_copy_service.cc:342] P 20730d75ad954a82b126148f8152a9f3: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-e42f46986036495ab1735d596ab06d6d received from {username='slave'} at 127.0.0.1:52758
I20260809 15:51:45.312104 10942 tablet_copy_service.cc:433] P 20730d75ad954a82b126148f8152a9f3: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-e42f46986036495ab1735d596ab06d6d on tablet e42f46986036495ab1735d596ab06d6d with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:45.326788 12010 tablet_copy_client.cc:791] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:45.327345 12010 tablet_copy_client.cc:655] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:45.330757 12010 tablet_copy_client.cc:531] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:45.338479 12010 tablet_bootstrap.cc:492] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:45.363653 12010 tablet_bootstrap.cc:492] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:45.364550 12010 tablet_bootstrap.cc:492] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:45.365193 12010 ts_tablet_manager.cc:1397] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.027s user 0.005s sys 0.014s
I20260809 15:51:45.368160 12010 raft_consensus.cc:348] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:45.368804 12010 raft_consensus.cc:729] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:45.369341 12010 consensus_queue.cc:260] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:45.379249 12007 ts_tablet_manager.cc:927] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:45.381407 12010 ts_tablet_manager.cc:1428] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.016s user 0.006s sys 0.000s
I20260809 15:51:45.387607 10942 tablet_copy_service.cc:342] P 20730d75ad954a82b126148f8152a9f3: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-11cab658217a44bebba11963c723e3dc received from {username='slave'} at 127.0.0.1:52758
I20260809 15:51:45.387954 10942 tablet_copy_service.cc:433] P 20730d75ad954a82b126148f8152a9f3: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-11cab658217a44bebba11963c723e3dc on tablet 11cab658217a44bebba11963c723e3dc with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:45.391839 12007 tablet_copy_client.cc:316] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.2:32783
I20260809 15:51:45.393281 10942 tablet_copy_service.cc:140] P 20730d75ad954a82b126148f8152a9f3: Received BeginTabletCopySession request for tablet 9c38f65ac0de428ba1d25c6f0bb6dc91 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:52758)
I20260809 15:51:45.393746 10942 tablet_copy_service.cc:161] P 20730d75ad954a82b126148f8152a9f3: Beginning new tablet copy session on tablet 9c38f65ac0de428ba1d25c6f0bb6dc91 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:52758: session id = c7aa762318cb4e97875333f10679a9f9-9c38f65ac0de428ba1d25c6f0bb6dc91
I20260809 15:51:45.400905 10942 tablet_copy_source_session.cc:205] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:45.405475 12007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9c38f65ac0de428ba1d25c6f0bb6dc91. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:45.424240 12007 tablet_copy_client.cc:791] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:45.424773 12007 tablet_copy_client.cc:655] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:45.428669 12007 tablet_copy_client.cc:531] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:45.438009 12007 tablet_bootstrap.cc:492] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:45.456017 12007 tablet_bootstrap.cc:492] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:45.456837 12007 tablet_bootstrap.cc:492] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:45.457494 12007 ts_tablet_manager.cc:1397] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.020s user 0.019s sys 0.000s
I20260809 15:51:45.460134 12007 raft_consensus.cc:348] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:45.460964 12007 raft_consensus.cc:729] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:45.461655 12007 consensus_queue.cc:260] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:45.465605 12007 ts_tablet_manager.cc:1428] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.008s user 0.009s sys 0.000s
I20260809 15:51:45.467163 10942 tablet_copy_service.cc:342] P 20730d75ad954a82b126148f8152a9f3: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-9c38f65ac0de428ba1d25c6f0bb6dc91 received from {username='slave'} at 127.0.0.1:52758
I20260809 15:51:45.467494 10942 tablet_copy_service.cc:433] P 20730d75ad954a82b126148f8152a9f3: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-9c38f65ac0de428ba1d25c6f0bb6dc91 on tablet 9c38f65ac0de428ba1d25c6f0bb6dc91 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:45.605481 11641 raft_consensus.cc:1203] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:45.710682 11641 raft_consensus.cc:1203] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:45.733569 11641 raft_consensus.cc:1203] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:45.823824 11641 raft_consensus.cc:1203] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:46.024891 11991 raft_consensus.cc:1050] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:46.027343 11991 consensus_queue.cc:237] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:46.040558 11639 raft_consensus.cc:1261] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:46.041406 11017 raft_consensus.cc:1261] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:46.044512 11820 consensus_queue.cc:1035] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:46.045905 11820 consensus_queue.cc:1035] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:46.054891 10843 raft_consensus.cc:1261] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:46.057273 11991 consensus_queue.cc:1035] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:46.067209 12018 raft_consensus.cc:2932] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.070837 11012 raft_consensus.cc:2932] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.077864 10843 raft_consensus.cc:2932] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.083143 11639 raft_consensus.cc:2932] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.089777 10733 catalog_manager.cc:5495] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:46.127470 10924 consensus_queue.cc:237] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:46.135272 11639 raft_consensus.cc:1261] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:46.135493 10843 raft_consensus.cc:1261] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:46.136802 11820 consensus_queue.cc:1035] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:46.137666 12018 consensus_queue.cc:1035] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:46.145248 12018 raft_consensus.cc:2932] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.149591 11639 raft_consensus.cc:2932] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.153046 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet e42f46986036495ab1735d596ab06d6d with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:46.162000 10736 catalog_manager.cc:5495] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.163838 10840 raft_consensus.cc:2932] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.173982 10998 tablet_service.cc:1514] Processing DeleteTablet for tablet e42f46986036495ab1735d596ab06d6d with delete_type TABLET_DATA_TOMBSTONED (TS d74944fa9fa14ec78b0640ea709ff3aa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50526
I20260809 15:51:46.182499 12027 tablet_replica.cc:330] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa: stopping tablet replica
I20260809 15:51:46.183871 12027 raft_consensus.cc:2229] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:46.187526 12027 raft_consensus.cc:2258] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:46.191154 12027 ts_tablet_manager.cc:1905] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:46.207655 12027 ts_tablet_manager.cc:1918] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:46.208057 12027 log.cc:1196] T e42f46986036495ab1735d596ab06d6d P d74944fa9fa14ec78b0640ea709ff3aa: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-2-root/wals/e42f46986036495ab1735d596ab06d6d
I20260809 15:51:46.209929 10719 catalog_manager.cc:4841] TS d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): tablet e42f46986036495ab1735d596ab06d6d (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:46.292096 11991 raft_consensus.cc:1050] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:46.294636 11991 consensus_queue.cc:237] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:46.304193 11012 raft_consensus.cc:1261] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:46.304704 11641 raft_consensus.cc:1261] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:46.305490 11991 consensus_queue.cc:1035] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:46.305913 10840 raft_consensus.cc:1261] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:46.307245 11820 consensus_queue.cc:1035] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:46.309095 12018 consensus_queue.cc:1035] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:46.322780 12015 raft_consensus.cc:1050] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:46.324793 12015 consensus_queue.cc:237] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:46.382174 12018 raft_consensus.cc:2932] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.389787 11639 raft_consensus.cc:2932] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.407176 10736 catalog_manager.cc:5495] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.425885 11991 raft_consensus.cc:1050] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:46.426333 11012 raft_consensus.cc:2932] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.429167 11991 consensus_queue.cc:237] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:46.442557 10840 raft_consensus.cc:2932] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.451202 10923 raft_consensus.cc:1261] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:46.451807 11641 raft_consensus.cc:1261] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:46.452526 10848 raft_consensus.cc:1261] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:46.452574 11980 consensus_queue.cc:1035] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:46.453622 12015 consensus_queue.cc:1035] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:46.456686 10923 consensus_queue.cc:237] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:46.459065 11641 raft_consensus.cc:1261] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:46.459223 11012 raft_consensus.cc:1261] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:46.460496 11991 consensus_queue.cc:1035] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:46.461381 11820 consensus_queue.cc:1035] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:46.462703 10848 raft_consensus.cc:1261] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:46.466455 12034 consensus_queue.cc:1035] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:46.467604 11639 raft_consensus.cc:1261] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:46.468964 12018 consensus_queue.cc:1035] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:46.485067 10848 raft_consensus.cc:1261] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:46.488379 12018 consensus_queue.cc:1035] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:46.491281 12037 consensus_queue.cc:1035] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:46.504129 12018 raft_consensus.cc:2932] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.506801 11820 raft_consensus.cc:2932] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.517561 10848 raft_consensus.cc:2932] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.513077 11641 raft_consensus.cc:2932] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.530148 11639 raft_consensus.cc:2932] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.534220 11980 raft_consensus.cc:2932] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.541227 10734 catalog_manager.cc:5495] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.544803 10848 raft_consensus.cc:2932] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.554966 11642 raft_consensus.cc:2932] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.556208 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 11cab658217a44bebba11963c723e3dc with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:46.584733 10733 catalog_manager.cc:5495] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:46.601009 10923 raft_consensus.cc:2932] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.583276 10843 raft_consensus.cc:2932] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.616307 10736 catalog_manager.cc:5495] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.597754 11012 raft_consensus.cc:2932] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.700743 11010 consensus_queue.cc:237] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:46.701968 10998 tablet_service.cc:1514] Processing DeleteTablet for tablet 11cab658217a44bebba11963c723e3dc with delete_type TABLET_DATA_TOMBSTONED (TS d74944fa9fa14ec78b0640ea709ff3aa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50526
I20260809 15:51:46.703711 12027 tablet_replica.cc:330] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa: stopping tablet replica
I20260809 15:51:46.717121 10924 consensus_queue.cc:237] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:46.731863 11639 raft_consensus.cc:1261] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:46.733417 12018 consensus_queue.cc:1035] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:46.734200 11011 raft_consensus.cc:1261] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:46.735994 12018 consensus_queue.cc:1035] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:46.744817 12027 raft_consensus.cc:2229] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:46.745507 12027 raft_consensus.cc:2258] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:46.749431 12027 ts_tablet_manager.cc:1905] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260809 15:51:46.750051 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 109728us
I20260809 15:51:46.748708 12037 raft_consensus.cc:2932] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.751735 11639 raft_consensus.cc:2932] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.754441 11010 raft_consensus.cc:2932] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.767134 10848 raft_consensus.cc:1261] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:46.769407 10733 catalog_manager.cc:5495] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.775128 11639 raft_consensus.cc:1261] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:46.778062 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 9c38f65ac0de428ba1d25c6f0bb6dc91 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:46.790043 10828 tablet_service.cc:1514] Processing DeleteTablet for tablet 9c38f65ac0de428ba1d25c6f0bb6dc91 with delete_type TABLET_DATA_TOMBSTONED (TS a53441872c0c416b8e2f26f52399f536 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45572
I20260809 15:51:46.804013 12048 consensus_queue.cc:1035] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:51:46.818511 11980 consensus_queue.cc:1035] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:46.820998 12049 tablet_replica.cc:330] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536: stopping tablet replica
I20260809 15:51:46.822464 12049 raft_consensus.cc:2229] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:46.847919 11641 raft_consensus.cc:2932] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.845942 12050 raft_consensus.cc:2932] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.841687 12027 ts_tablet_manager.cc:1918] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:46.852836 12027 log.cc:1196] T 11cab658217a44bebba11963c723e3dc P d74944fa9fa14ec78b0640ea709ff3aa: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-2-root/wals/11cab658217a44bebba11963c723e3dc
I20260809 15:51:46.853956 10840 raft_consensus.cc:2932] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.855445 12049 raft_consensus.cc:2258] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:46.857383 10719 catalog_manager.cc:4841] TS d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): tablet 11cab658217a44bebba11963c723e3dc (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:46.864537 12049 ts_tablet_manager.cc:1905] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:46.864925 10719 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 2b4306bfa5464b0dbf8301756455ba50 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:46.866910 10736 catalog_manager.cc:5495] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:46.888726 12049 ts_tablet_manager.cc:1918] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:46.889353 12049 log.cc:1196] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P a53441872c0c416b8e2f26f52399f536: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-0-root/wals/9c38f65ac0de428ba1d25c6f0bb6dc91
I20260809 15:51:46.891850 10722 catalog_manager.cc:4841] TS a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): tablet 9c38f65ac0de428ba1d25c6f0bb6dc91 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:46.894819 10913 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b4306bfa5464b0dbf8301756455ba50 with delete_type TABLET_DATA_TOMBSTONED (TS 20730d75ad954a82b126148f8152a9f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54216
I20260809 15:51:46.896754 12052 tablet_replica.cc:330] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3: stopping tablet replica
I20260809 15:51:46.899698 12052 raft_consensus.cc:2229] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:46.900528 12052 raft_consensus.cc:2258] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:46.904718 12052 ts_tablet_manager.cc:1905] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:46.915997 12052 ts_tablet_manager.cc:1918] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:46.916299 12052 log.cc:1196] T 2b4306bfa5464b0dbf8301756455ba50 P 20730d75ad954a82b126148f8152a9f3: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-1-root/wals/2b4306bfa5464b0dbf8301756455ba50
I20260809 15:51:46.917572 10722 catalog_manager.cc:4841] TS 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): tablet 2b4306bfa5464b0dbf8301756455ba50 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:47.940569 11017 consensus_queue.cc:237] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:47.947779 10923 raft_consensus.cc:1261] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 15:51:47.947892 10978 consensus_peers.cc:487] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6508f17228904310be3c95818973f3a1. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:47.949159 10843 raft_consensus.cc:1261] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:47.949740 12050 consensus_queue.cc:1035] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:47.953565 12050 consensus_queue.cc:1035] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:47.960595 12015 raft_consensus.cc:2932] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:47.962855 10923 raft_consensus.cc:2932] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:47.964241 10843 raft_consensus.cc:2932] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:47.984045 11017 consensus_queue.cc:237] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:47.984652 10737 catalog_manager.cc:5495] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:47.994155 10843 raft_consensus.cc:1261] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:47.994390 10923 raft_consensus.cc:1261] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 15:51:47.994952 10978 consensus_peers.cc:487] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5dc95cd5fbb945e7af0f89cf2ec436c9. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:47.996613 12061 consensus_queue.cc:1035] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:47.998355 12056 consensus_queue.cc:1035] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:48.023283 12061 raft_consensus.cc:2932] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:48.033686 10848 raft_consensus.cc:2932] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:48.040148 10923 raft_consensus.cc:2932] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:48.042049 10840 consensus_queue.cc:237] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:48.055892 10736 catalog_manager.cc:5495] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:48.065583 10923 raft_consensus.cc:1261] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 15:51:48.067121 10811 consensus_peers.cc:487] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 2edb972a2a164f00a942aa6248187fe8. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:48.067227 11017 raft_consensus.cc:1261] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:48.068111 11982 consensus_queue.cc:1035] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:48.069027 12072 consensus_queue.cc:1035] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:48.091753 11982 raft_consensus.cc:2932] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:48.093978 10931 raft_consensus.cc:2932] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:48.099901 11010 raft_consensus.cc:2932] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:48.125952 10733 catalog_manager.cc:5495] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:48.127346 10923 consensus_queue.cc:237] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:48.139923 11017 raft_consensus.cc:1261] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 15:51:48.186313 10893 consensus_peers.cc:487] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 02f25d93293a40729fd751a19ab9adeb. This is attempt 1: this message will repeat every 5th retry.
W20260809 15:51:48.187678 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 48058.3us
I20260809 15:51:48.191821 10848 raft_consensus.cc:1261] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:48.200032 12081 consensus_queue.cc:1035] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260809 15:51:48.247323 12053 raft_consensus.cc:2932] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:48.253618 11011 raft_consensus.cc:2932] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
W20260809 15:51:48.260437 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 66936us
I20260809 15:51:48.266994 12081 consensus_queue.cc:1035] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:48.308668 10840 raft_consensus.cc:2932] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:48.312705 10737 catalog_manager.cc:5495] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:48.440125 12085 ts_tablet_manager.cc:927] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:48.442765 12085 tablet_copy_client.cc:316] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.3:39223
I20260809 15:51:48.444787 11027 tablet_copy_service.cc:140] P d74944fa9fa14ec78b0640ea709ff3aa: Received BeginTabletCopySession request for tablet 5dc95cd5fbb945e7af0f89cf2ec436c9 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:39978)
I20260809 15:51:48.445268 11027 tablet_copy_service.cc:161] P d74944fa9fa14ec78b0640ea709ff3aa: Beginning new tablet copy session on tablet 5dc95cd5fbb945e7af0f89cf2ec436c9 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:39978: session id = c7aa762318cb4e97875333f10679a9f9-5dc95cd5fbb945e7af0f89cf2ec436c9
I20260809 15:51:48.452818 11027 tablet_copy_source_session.cc:205] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:48.455857 12085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5dc95cd5fbb945e7af0f89cf2ec436c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:48.470276 12085 tablet_copy_client.cc:791] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:48.470846 12085 tablet_copy_client.cc:655] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:48.474292 12085 tablet_copy_client.cc:531] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:48.480876 12085 tablet_bootstrap.cc:492] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:48.501546 12087 ts_tablet_manager.cc:927] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:48.503839 12087 tablet_copy_client.cc:316] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.3:39223
I20260809 15:51:48.510633 12085 tablet_bootstrap.cc:492] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:48.512454 12085 tablet_bootstrap.cc:492] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:48.512682 11027 tablet_copy_service.cc:140] P d74944fa9fa14ec78b0640ea709ff3aa: Received BeginTabletCopySession request for tablet 6508f17228904310be3c95818973f3a1 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:39978)
I20260809 15:51:48.513149 12085 ts_tablet_manager.cc:1397] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.033s user 0.018s sys 0.005s
I20260809 15:51:48.513252 11027 tablet_copy_service.cc:161] P d74944fa9fa14ec78b0640ea709ff3aa: Beginning new tablet copy session on tablet 6508f17228904310be3c95818973f3a1 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:39978: session id = c7aa762318cb4e97875333f10679a9f9-6508f17228904310be3c95818973f3a1
I20260809 15:51:48.516503 12085 raft_consensus.cc:348] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:48.517400 12085 raft_consensus.cc:729] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:48.518038 12085 consensus_queue.cc:260] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:48.521428 11027 tablet_copy_source_session.cc:205] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:48.524049 12085 ts_tablet_manager.cc:1428] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.010s user 0.004s sys 0.005s
I20260809 15:51:48.527673 12087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6508f17228904310be3c95818973f3a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:48.540727 11027 tablet_copy_service.cc:342] P d74944fa9fa14ec78b0640ea709ff3aa: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-5dc95cd5fbb945e7af0f89cf2ec436c9 received from {username='slave'} at 127.0.0.1:39978
I20260809 15:51:48.541184 11027 tablet_copy_service.cc:433] P d74944fa9fa14ec78b0640ea709ff3aa: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-5dc95cd5fbb945e7af0f89cf2ec436c9 on tablet 5dc95cd5fbb945e7af0f89cf2ec436c9 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:48.545326 12087 tablet_copy_client.cc:791] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:48.545863 12087 tablet_copy_client.cc:655] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:48.547493 12085 ts_tablet_manager.cc:927] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:48.554373 12087 tablet_copy_client.cc:531] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:48.557837 12085 tablet_copy_client.cc:316] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.2:32783
I20260809 15:51:48.560487 10942 tablet_copy_service.cc:140] P 20730d75ad954a82b126148f8152a9f3: Received BeginTabletCopySession request for tablet 02f25d93293a40729fd751a19ab9adeb from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:52758)
I20260809 15:51:48.560979 10942 tablet_copy_service.cc:161] P 20730d75ad954a82b126148f8152a9f3: Beginning new tablet copy session on tablet 02f25d93293a40729fd751a19ab9adeb from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:52758: session id = c7aa762318cb4e97875333f10679a9f9-02f25d93293a40729fd751a19ab9adeb
I20260809 15:51:48.568054 12087 tablet_bootstrap.cc:492] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:48.568432 10942 tablet_copy_source_session.cc:205] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:48.570932 12085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02f25d93293a40729fd751a19ab9adeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:48.587100 12091 ts_tablet_manager.cc:927] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615)
I20260809 15:51:48.589260 12085 tablet_copy_client.cc:791] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:48.589819 12085 tablet_copy_client.cc:655] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:48.590776 12087 tablet_bootstrap.cc:492] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:48.591522 12087 tablet_bootstrap.cc:492] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:48.592219 12087 ts_tablet_manager.cc:1397] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.024s user 0.016s sys 0.008s
I20260809 15:51:48.607019 12085 tablet_copy_client.cc:531] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:48.609591 12087 raft_consensus.cc:348] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:48.610503 12087 raft_consensus.cc:729] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:48.611136 12087 consensus_queue.cc:260] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:48.616066 12087 ts_tablet_manager.cc:1428] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.024s user 0.004s sys 0.004s
I20260809 15:51:48.616451 12091 tablet_copy_client.cc:316] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.1:46615
I20260809 15:51:48.618436 10858 tablet_copy_service.cc:140] P a53441872c0c416b8e2f26f52399f536: Received BeginTabletCopySession request for tablet 2edb972a2a164f00a942aa6248187fe8 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:54342)
I20260809 15:51:48.618965 10858 tablet_copy_service.cc:161] P a53441872c0c416b8e2f26f52399f536: Beginning new tablet copy session on tablet 2edb972a2a164f00a942aa6248187fe8 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:54342: session id = c7aa762318cb4e97875333f10679a9f9-2edb972a2a164f00a942aa6248187fe8
I20260809 15:51:48.619354 12085 tablet_bootstrap.cc:492] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:48.620163 11027 tablet_copy_service.cc:342] P d74944fa9fa14ec78b0640ea709ff3aa: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-6508f17228904310be3c95818973f3a1 received from {username='slave'} at 127.0.0.1:39978
I20260809 15:51:48.620568 11027 tablet_copy_service.cc:433] P d74944fa9fa14ec78b0640ea709ff3aa: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-6508f17228904310be3c95818973f3a1 on tablet 6508f17228904310be3c95818973f3a1 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:48.633366 10858 tablet_copy_source_session.cc:205] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:48.640837 12091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2edb972a2a164f00a942aa6248187fe8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:48.656744 12091 tablet_copy_client.cc:791] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:48.657351 12091 tablet_copy_client.cc:655] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:48.658264 12085 tablet_bootstrap.cc:492] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:48.659233 12085 tablet_bootstrap.cc:492] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:48.659893 12085 ts_tablet_manager.cc:1397] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.041s user 0.022s sys 0.003s
I20260809 15:51:48.663074 12085 raft_consensus.cc:348] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:48.663858 12085 raft_consensus.cc:729] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:48.664535 12085 consensus_queue.cc:260] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:48.669991 12085 ts_tablet_manager.cc:1428] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.010s user 0.007s sys 0.000s
I20260809 15:51:48.671509 10942 tablet_copy_service.cc:342] P 20730d75ad954a82b126148f8152a9f3: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-02f25d93293a40729fd751a19ab9adeb received from {username='slave'} at 127.0.0.1:52758
I20260809 15:51:48.671877 10942 tablet_copy_service.cc:433] P 20730d75ad954a82b126148f8152a9f3: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-02f25d93293a40729fd751a19ab9adeb on tablet 02f25d93293a40729fd751a19ab9adeb with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:48.676085 12091 tablet_copy_client.cc:531] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:48.684923 12091 tablet_bootstrap.cc:492] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:48.704885 12091 tablet_bootstrap.cc:492] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:48.705547 12091 tablet_bootstrap.cc:492] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:48.706166 12091 ts_tablet_manager.cc:1397] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.021s user 0.019s sys 0.004s
I20260809 15:51:48.708544 12091 raft_consensus.cc:348] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:48.709197 12091 raft_consensus.cc:729] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:48.709777 12091 consensus_queue.cc:260] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:48.711709 12091 ts_tablet_manager.cc:1428] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 15:51:48.713778 10858 tablet_copy_service.cc:342] P a53441872c0c416b8e2f26f52399f536: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-2edb972a2a164f00a942aa6248187fe8 received from {username='slave'} at 127.0.0.1:54342
I20260809 15:51:48.718207 10858 tablet_copy_service.cc:433] P a53441872c0c416b8e2f26f52399f536: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-2edb972a2a164f00a942aa6248187fe8 on tablet 2edb972a2a164f00a942aa6248187fe8 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:48.871603 11639 raft_consensus.cc:1203] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:48.911898 11639 raft_consensus.cc:1203] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:49.043721 11641 raft_consensus.cc:1203] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:49.208024 11641 raft_consensus.cc:1203] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:49.319798 12060 raft_consensus.cc:1050] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:49.322306 12060 consensus_queue.cc:237] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:49.330067 11641 raft_consensus.cc:1261] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:49.335204 12054 consensus_queue.cc:1035] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:49.399133 10927 raft_consensus.cc:1261] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:49.401007 12054 consensus_queue.cc:1035] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:49.450700 10843 raft_consensus.cc:1261] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
W20260809 15:51:49.510581 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 102520us
I20260809 15:51:49.523298 12054 consensus_queue.cc:1035] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:49.536846 10927 raft_consensus.cc:2932] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.530154 12110 raft_consensus.cc:2932] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.548135 12106 raft_consensus.cc:1050] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:49.550748 12106 consensus_queue.cc:237] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:49.567130 11642 raft_consensus.cc:2932] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.574450 12050 raft_consensus.cc:1050] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:49.576814 12050 consensus_queue.cc:237] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:49.573899 10736 catalog_manager.cc:5495] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.577850 10843 raft_consensus.cc:2932] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.617178 11639 raft_consensus.cc:1261] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:49.624605 12110 consensus_queue.cc:1035] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:49.625443 11642 raft_consensus.cc:1261] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:49.625648 10926 raft_consensus.cc:1261] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:49.627089 12106 consensus_queue.cc:1035] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:49.628463 11011 raft_consensus.cc:1261] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:49.628926 10844 raft_consensus.cc:1261] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:49.630060 12106 consensus_queue.cc:1035] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:49.633803 10843 raft_consensus.cc:1261] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:49.637158 12106 consensus_queue.cc:1035] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:49.639560 12056 consensus_queue.cc:1035] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:49.653818 11642 raft_consensus.cc:2932] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.656103 12056 consensus_queue.cc:1035] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:49.652336 12106 raft_consensus.cc:2932] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.677141 10843 raft_consensus.cc:2932] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.677480 12110 raft_consensus.cc:2932] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.681725 11018 raft_consensus.cc:2932] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.688194 11642 raft_consensus.cc:2932] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.695570 10926 raft_consensus.cc:2932] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.719295 10848 raft_consensus.cc:2932] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.728055 11011 consensus_queue.cc:237] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:49.735898 10734 catalog_manager.cc:5495] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.737533 10734 catalog_manager.cc:5495] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.745538 10926 raft_consensus.cc:1261] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:49.751660 11639 raft_consensus.cc:1261] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:49.751551 12110 consensus_queue.cc:1035] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:51:49.753105 12056 consensus_queue.cc:1035] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:49.762571 12056 raft_consensus.cc:2932] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.775393 11639 raft_consensus.cc:2932] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.778858 11011 consensus_queue.cc:237] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:49.785245 10719 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 5dc95cd5fbb945e7af0f89cf2ec436c9 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:49.787627 10923 consensus_queue.cc:237] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:49.791301 10926 raft_consensus.cc:2932] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.793426 11642 raft_consensus.cc:1261] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:49.792176 10843 raft_consensus.cc:1261] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:49.797327 10843 raft_consensus.cc:1261] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:49.797377 11642 raft_consensus.cc:1261] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:49.798949 12050 consensus_queue.cc:1035] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:49.801131 12109 consensus_queue.cc:1035] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:49.805619 12107 consensus_queue.cc:1035] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:51:49.806180 10736 catalog_manager.cc:5495] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:49.807229 12050 raft_consensus.cc:2932] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.808094 12107 consensus_queue.cc:1035] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:49.816069 12101 raft_consensus.cc:1050] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:49.818625 12101 consensus_queue.cc:237] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:49.825606 11642 raft_consensus.cc:2932] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.827705 12106 raft_consensus.cc:2932] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.833827 10828 tablet_service.cc:1514] Processing DeleteTablet for tablet 5dc95cd5fbb945e7af0f89cf2ec436c9 with delete_type TABLET_DATA_TOMBSTONED (TS a53441872c0c416b8e2f26f52399f536 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45572
I20260809 15:51:49.847474 10848 raft_consensus.cc:2932] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.853323 10733 catalog_manager.cc:5495] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.852712 10843 raft_consensus.cc:2932] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.863960 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 02f25d93293a40729fd751a19ab9adeb with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:49.843187 11642 raft_consensus.cc:2932] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:49.881224 10913 tablet_service.cc:1514] Processing DeleteTablet for tablet 6508f17228904310be3c95818973f3a1 with delete_type TABLET_DATA_TOMBSTONED (TS 20730d75ad954a82b126148f8152a9f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54216
I20260809 15:51:49.883750 10719 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 6508f17228904310be3c95818973f3a1 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:49.890061 10736 catalog_manager.cc:5495] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 3 to 4, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:49.907063 10998 tablet_service.cc:1514] Processing DeleteTablet for tablet 02f25d93293a40729fd751a19ab9adeb with delete_type TABLET_DATA_TOMBSTONED (TS d74944fa9fa14ec78b0640ea709ff3aa not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:50526
I20260809 15:51:49.934165 11011 raft_consensus.cc:1261] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:49.935640 10926 raft_consensus.cc:1261] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:49.935767 11642 raft_consensus.cc:1261] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:49.938112 12127 tablet_replica.cc:330] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536: stopping tablet replica
I20260809 15:51:49.939517 12127 raft_consensus.cc:2229] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:49.940511 12127 raft_consensus.cc:2258] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:49.947999 12099 consensus_queue.cc:1035] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:49.952662 12101 consensus_queue.cc:1035] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:49.966907 12098 consensus_queue.cc:1035] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.009s
I20260809 15:51:49.982664 12128 tablet_replica.cc:330] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3: stopping tablet replica
I20260809 15:51:49.985488 12127 ts_tablet_manager.cc:1905] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:49.986406 12128 raft_consensus.cc:2229] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:49.993770 12129 tablet_replica.cc:330] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa: stopping tablet replica
I20260809 15:51:49.995656 12129 raft_consensus.cc:2229] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:49.992213 12128 raft_consensus.cc:2258] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:50.018291 12128 ts_tablet_manager.cc:1905] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:50.013710 12129 raft_consensus.cc:2258] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:50.029110 12129 ts_tablet_manager.cc:1905] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:50.033077 12078 raft_consensus.cc:2932] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:50.037250 10927 raft_consensus.cc:2932] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:50.040563 11018 raft_consensus.cc:2932] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:50.055660 12127 ts_tablet_manager.cc:1918] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:50.056299 12127 log.cc:1196] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P a53441872c0c416b8e2f26f52399f536: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-0-root/wals/5dc95cd5fbb945e7af0f89cf2ec436c9
I20260809 15:51:50.058454 10722 catalog_manager.cc:4841] TS a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): tablet 5dc95cd5fbb945e7af0f89cf2ec436c9 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:50.063297 10734 catalog_manager.cc:5495] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:50.066229 12129 ts_tablet_manager.cc:1918] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:50.066967 12129 log.cc:1196] T 02f25d93293a40729fd751a19ab9adeb P d74944fa9fa14ec78b0640ea709ff3aa: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-2-root/wals/02f25d93293a40729fd751a19ab9adeb
I20260809 15:51:50.069612 10719 catalog_manager.cc:4841] TS d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): tablet 02f25d93293a40729fd751a19ab9adeb (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:50.082623 12128 ts_tablet_manager.cc:1918] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:50.083325 12128 log.cc:1196] T 6508f17228904310be3c95818973f3a1 P 20730d75ad954a82b126148f8152a9f3: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-1-root/wals/6508f17228904310be3c95818973f3a1
I20260809 15:51:50.086170 10722 catalog_manager.cc:4841] TS 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): tablet 6508f17228904310be3c95818973f3a1 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:50.089562 11642 raft_consensus.cc:2932] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:50.096421 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:50.114359 10840 consensus_queue.cc:237] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:50.120234 11639 raft_consensus.cc:1261] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:50.121769 12101 consensus_queue.cc:1035] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:51:50.124141 11011 raft_consensus.cc:1261] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:50.137156 11641 raft_consensus.cc:2932] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:50.144364 12101 raft_consensus.cc:2932] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:50.149654 12078 consensus_queue.cc:1035] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:51:50.154166 11011 raft_consensus.cc:2932] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:50.162473 10734 catalog_manager.cc:5495] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:50.165586 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 2edb972a2a164f00a942aa6248187fe8 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:50.178171 10913 tablet_service.cc:1514] Processing DeleteTablet for tablet 2edb972a2a164f00a942aa6248187fe8 with delete_type TABLET_DATA_TOMBSTONED (TS 20730d75ad954a82b126148f8152a9f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54216
I20260809 15:51:50.179450 12128 tablet_replica.cc:330] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3: stopping tablet replica
I20260809 15:51:50.190871 12128 raft_consensus.cc:2229] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:50.191828 12128 raft_consensus.cc:2258] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:50.195986 12128 ts_tablet_manager.cc:1905] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:50.225922 12128 ts_tablet_manager.cc:1918] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:50.226465 12128 log.cc:1196] T 2edb972a2a164f00a942aa6248187fe8 P 20730d75ad954a82b126148f8152a9f3: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-1-root/wals/2edb972a2a164f00a942aa6248187fe8
I20260809 15:51:50.228642 10722 catalog_manager.cc:4841] TS 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): tablet 2edb972a2a164f00a942aa6248187fe8 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:50.234226 10927 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "11cab658217a44bebba11963c723e3dc"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:51:50.234913 10927 raft_consensus.cc:595] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Received request to transfer leadership to TS c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:50.255064 11011 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:51:50.255724 11011 raft_consensus.cc:595] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Received request to transfer leadership to TS c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:50.258426 10927 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "220f58b662044f089f2be802ed24284a"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:51:50.258865 10927 raft_consensus.cc:595] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:51:50.261818 10927 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:51:50.262279 10927 raft_consensus.cc:595] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Received request to transfer leadership to TS c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:50.280530 10840 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:51:50.281028 10840 raft_consensus.cc:595] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:50.283133 10927 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:51:50.283639 10927 raft_consensus.cc:595] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Received request to transfer leadership to TS c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:50.285593 10927 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:51:50.286011 10927 raft_consensus.cc:595] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Received request to transfer leadership to TS c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:50.287092 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 7
I20260809 15:51:50.288640 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:51:50.289283 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:50.419893 12081 raft_consensus.cc:979] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:51:50.420461 12081 raft_consensus.cc:1067] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Signalling peer c7aa762318cb4e97875333f10679a9f9 to start an election
W20260809 15:51:50.421320 10956 debug-util.cc:398] Leaking SignalData structure 0x7b08000a5420 after lost signal to thread 9556
I20260809 15:51:50.421892 11642 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4"
dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:41488
I20260809 15:51:50.422475 11642 raft_consensus.cc:482] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
W20260809 15:51:50.422482 10956 debug-util.cc:398] Leaking SignalData structure 0x7b08000a5b00 after lost signal to thread 10773
I20260809 15:51:50.422851 11642 raft_consensus.cc:3037] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
W20260809 15:51:50.425722 10956 debug-util.cc:398] Leaking SignalData structure 0x7b0800340f00 after lost signal to thread 11667
I20260809 15:51:50.428678 11642 raft_consensus.cc:504] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:50.430547 11642 leader_election.cc:288] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:50.431219 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:50.431579 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:50.431941 10927 raft_consensus.cc:3032] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:50.432130 11011 raft_consensus.cc:3037] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:50.432329 10927 raft_consensus.cc:729] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:50.433185 10927 consensus_queue.cc:260] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:50.434397 10927 raft_consensus.cc:3037] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:50.437865 11011 raft_consensus.cc:2454] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:51:50.439791 11604 leader_election.cc:302] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:50.440716 12090 raft_consensus.cc:2781] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:50.449203 10927 raft_consensus.cc:2454] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:51:50.457427 12090 raft_consensus.cc:686] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:51:50.462520 12090 consensus_queue.cc:237] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:50.471589 10737 catalog_manager.cc:5495] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 1 to 2, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 2 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:50.561241 12110 raft_consensus.cc:979] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:51:50.561761 12056 raft_consensus.cc:1067] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Signalling peer c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:51:50.563109 11642 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1"
dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:41478
I20260809 15:51:50.563556 11642 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:50.563911 11642 raft_consensus.cc:3037] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:50.569674 11642 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:50.573529 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:50.574108 10840 raft_consensus.cc:3037] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:50.574538 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:50.575109 11011 raft_consensus.cc:3032] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:50.575363 11011 raft_consensus.cc:729] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:50.576030 11011 consensus_queue.cc:260] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:50.577042 11011 raft_consensus.cc:3037] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:50.579452 11642 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:50.582355 10840 raft_consensus.cc:2454] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:51:50.583669 11606 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:51:50.584492 12090 raft_consensus.cc:2781] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:50.584983 12090 raft_consensus.cc:686] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:51:50.585652 11011 raft_consensus.cc:2454] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:51:50.585858 12090 consensus_queue.cc:237] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:50.596545 10737 catalog_manager.cc:5495] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 1 to 2, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 2 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:50.614542 12078 raft_consensus.cc:979] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:51:50.615052 12098 raft_consensus.cc:1067] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:51:50.616349 10926 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54224
I20260809 15:51:50.616844 10926 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:50.617147 10926 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:50.622423 10926 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:50.624240 10926 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:50.624826 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:50.625433 10840 raft_consensus.cc:3032] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:50.625725 10840 raft_consensus.cc:729] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:50.626056 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:50.626843 11011 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:50.627525 10840 consensus_queue.cc:260] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:50.634603 10840 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:50.640341 11011 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:51:50.640595 10840 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:51:50.641672 10891 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:50.642454 12081 raft_consensus.cc:2781] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:50.643069 12081 raft_consensus.cc:686] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:50.643869 12081 consensus_queue.cc:237] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:50.655985 10737 catalog_manager.cc:5495] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 1 to 2, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:50.671387 12107 raft_consensus.cc:979] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:51:50.671845 12081 raft_consensus.cc:1067] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Signalling peer c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:51:50.673262 11642 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1"
dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:41488
I20260809 15:51:50.673775 11642 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:50.674049 11642 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:50.678284 11642 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:50.679965 11642 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:50.680946 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:50.681552 10840 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:50.683845 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:50.684631 10926 raft_consensus.cc:3032] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:50.686117 10926 raft_consensus.cc:729] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:50.686919 10926 consensus_queue.cc:260] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:50.687661 10840 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:51:50.688194 10926 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:50.688573 11606 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:51:50.689363 12090 raft_consensus.cc:2781] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:50.689788 12090 raft_consensus.cc:686] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:51:50.690565 12090 consensus_queue.cc:237] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:50.701653 10926 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:51:50.707223 10737 catalog_manager.cc:5495] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 1 to 2, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 2 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:50.765250 12107 raft_consensus.cc:979] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:51:50.765722 12081 raft_consensus.cc:1067] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:51:50.767249 10840 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "220f58b662044f089f2be802ed24284a"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45594
I20260809 15:51:50.767706 10840 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:50.768038 10840 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:50.774307 10840 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:50.776747 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:50.776922 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:50.777316 10926 raft_consensus.cc:3032] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:50.777489 11018 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:50.777731 10926 raft_consensus.cc:729] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:50.778560 10926 consensus_queue.cc:260] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:50.779713 10926 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:50.786511 10926 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:51:50.787634 11018 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:51:50.788039 10811 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:50.788579 10840 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:50.788805 12101 raft_consensus.cc:2781] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:50.789368 12101 raft_consensus.cc:686] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:50.790278 12101 consensus_queue.cc:237] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:50.791210 12107 raft_consensus.cc:979] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3: : Instructing follower c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:51:50.791630 12081 raft_consensus.cc:1067] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Signalling peer c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:51:50.793042 11642 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "11cab658217a44bebba11963c723e3dc"
dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:41488
I20260809 15:51:50.793462 11642 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:50.793848 11642 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:50.798835 11642 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:50.800319 11642 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:50.802698 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:50.803217 10926 raft_consensus.cc:3032] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:50.803490 10926 raft_consensus.cc:729] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:50.804175 10926 consensus_queue.cc:260] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:50.805246 12105 raft_consensus.cc:979] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:51:50.805408 10926 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:50.805720 12105 raft_consensus.cc:1067] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Signalling peer c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:51:50.812088 11642 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91"
dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:41488
I20260809 15:51:50.812557 11642 raft_consensus.cc:482] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:50.817968 10926 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:51:50.812893 11642 raft_consensus.cc:3037] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:50.819614 11606 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:51:50.820482 12090 raft_consensus.cc:2781] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:50.820919 12090 raft_consensus.cc:686] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:51:50.821736 12090 consensus_queue.cc:237] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:50.818460 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:50.835022 10840 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:50.836290 10734 catalog_manager.cc:5495] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 1 to 2, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 2 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:50.849628 11642 raft_consensus.cc:504] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:50.868008 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:50.868757 10926 raft_consensus.cc:3032] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:50.869038 10926 raft_consensus.cc:729] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:50.871217 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:50.869519 11642 leader_election.cc:288] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:50.871909 11018 raft_consensus.cc:3037] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:50.870502 10926 consensus_queue.cc:260] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:50.882061 10926 raft_consensus.cc:3037] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:50.882843 10734 catalog_manager.cc:5495] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 1 to 2, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:50.888643 11018 raft_consensus.cc:2454] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:51:50.889858 11604 leader_election.cc:302] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:50.890796 12090 raft_consensus.cc:2781] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:50.884582 10840 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:51:50.891247 12090 raft_consensus.cc:686] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:51:50.892089 12090 consensus_queue.cc:237] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:50.894490 10926 raft_consensus.cc:2454] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:51:50.906265 10734 catalog_manager.cc:5495] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 1 to 2, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 2 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:50.984093 10926 raft_consensus.cc:1261] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:50.985349 12090 consensus_queue.cc:1035] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:51.011019 11018 raft_consensus.cc:1261] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:51.012634 12139 consensus_queue.cc:1035] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:51.035830 11018 raft_consensus.cc:1261] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:51.037322 12151 consensus_queue.cc:1035] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:51.054332 10840 raft_consensus.cc:1261] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:51.055541 12151 consensus_queue.cc:1035] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:51.104311 10840 raft_consensus.cc:1261] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:51.107962 12107 consensus_queue.cc:1035] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:51.127329 11018 raft_consensus.cc:1261] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:51.131341 12082 consensus_queue.cc:1035] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:51.214828 10843 raft_consensus.cc:1261] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:51.216744 12151 consensus_queue.cc:1035] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:51.235782 10927 raft_consensus.cc:1261] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:51.238242 12090 consensus_queue.cc:1035] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:51.238557 11011 raft_consensus.cc:1261] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:51.239789 12101 consensus_queue.cc:1035] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:51.244015 10923 raft_consensus.cc:1261] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:51.248297 12090 consensus_queue.cc:1035] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:51.272984 10840 raft_consensus.cc:1261] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260809 15:51:51.273769 10789 auto_rebalancer.cc:254] failed to send replica move request: Illegal state: Leader has not yet committed an operation in its own term
I20260809 15:51:51.284482 12139 consensus_queue.cc:1035] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:51.307662 10923 raft_consensus.cc:1261] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:51.311935 12101 consensus_queue.cc:1035] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:51.330155 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:51.344595 11018 raft_consensus.cc:1261] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:51.353039 12151 consensus_queue.cc:1035] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:51.369275 10926 raft_consensus.cc:1261] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260809 15:51:51.369390 10789 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 220f58b662044f089f2be802ed24284a, TS d74944fa9fa14ec78b0640ea709ff3aa -> TS c7aa762318cb4e97875333f10679a9f9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:51.369741 10789 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 220f58b662044f089f2be802ed24284a, TS d74944fa9fa14ec78b0640ea709ff3aa -> TS c7aa762318cb4e97875333f10679a9f9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:51.386687 10789 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 6d4d6dee78914014adab40a391b02955, TS 20730d75ad954a82b126148f8152a9f3 -> TS c7aa762318cb4e97875333f10679a9f9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:51.387085 10789 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 6d4d6dee78914014adab40a391b02955, TS 20730d75ad954a82b126148f8152a9f3 -> TS c7aa762318cb4e97875333f10679a9f9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:51.390585 12151 consensus_queue.cc:1035] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:51.425480 10924 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:34958
W20260809 15:51:51.425787 10789 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 855d87c571d0427ba84900052dd9e2c0, TS a53441872c0c416b8e2f26f52399f536 -> TS c7aa762318cb4e97875333f10679a9f9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:51.426191 10789 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 855d87c571d0427ba84900052dd9e2c0, TS a53441872c0c416b8e2f26f52399f536 -> TS c7aa762318cb4e97875333f10679a9f9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:51.426105 10924 raft_consensus.cc:595] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Received request to transfer leadership to TS c7aa762318cb4e97875333f10679a9f9
W20260809 15:51:51.442823 10789 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet e5757bc9ac9441c3bdfa6c5b2b743b6d, TS 20730d75ad954a82b126148f8152a9f3 -> TS c7aa762318cb4e97875333f10679a9f9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:51.443421 10789 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet e5757bc9ac9441c3bdfa6c5b2b743b6d, TS 20730d75ad954a82b126148f8152a9f3 -> TS c7aa762318cb4e97875333f10679a9f9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:51.571816 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 101922us
W20260809 15:51:51.572505 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 50308.2us
I20260809 15:51:51.581048 11011 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:51:51.581619 11011 raft_consensus.cc:595] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
W20260809 15:51:51.582793 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 83180.6us
I20260809 15:51:51.601914 12174 raft_consensus.cc:979] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3: : Instructing follower c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:51:51.602391 12174 raft_consensus.cc:1067] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Signalling peer c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:51:51.603785 11641 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d"
dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:41488
I20260809 15:51:51.604278 11641 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:51.604580 11641 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:51.607641 10927 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:51:51.608587 10927 raft_consensus.cc:595] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:51:51.612702 10848 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:51:51.613234 10848 raft_consensus.cc:595] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Received request to transfer leadership to TS c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:51.615630 10848 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:51:51.616142 10848 raft_consensus.cc:595] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:51.617398 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260809 15:51:51.619040 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:51:51.619727 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:51.613195 11641 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:51.621862 11641 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:51.622767 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:51.627359 10848 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:51.623273 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:51.633925 10923 raft_consensus.cc:3032] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:51.634234 10923 raft_consensus.cc:729] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:51.634938 10923 consensus_queue.cc:260] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:51.636070 10923 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:51.638188 10848 raft_consensus.cc:2454] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:51:51.640446 11606 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:51:51.642830 10923 raft_consensus.cc:2454] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:51:51.646251 12139 raft_consensus.cc:2781] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:51.646733 12139 raft_consensus.cc:686] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:51:51.647373 12139 consensus_queue.cc:237] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:51.675395 10734 catalog_manager.cc:5495] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 1 to 2, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 2 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:51.682899 12101 raft_consensus.cc:979] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:51:51.683470 12098 raft_consensus.cc:1067] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:51:51.684942 10923 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54224
I20260809 15:51:51.685443 10923 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:51.685761 10923 raft_consensus.cc:3037] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:51.693305 10923 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:51.696260 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:51.696770 10848 raft_consensus.cc:3032] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:51.697065 10848 raft_consensus.cc:729] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:51.697398 10923 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:51.697937 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:51.697680 10848 consensus_queue.cc:260] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:51.698546 11018 raft_consensus.cc:3037] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:51.699363 10848 raft_consensus.cc:3037] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:51.706539 11018 raft_consensus.cc:2454] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:51:51.711012 12101 raft_consensus.cc:979] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536: : Instructing follower c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:51:51.711076 10848 raft_consensus.cc:2454] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:51:51.711510 10891 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:51.711658 12101 raft_consensus.cc:1067] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Signalling peer c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:51:51.712459 12178 raft_consensus.cc:2781] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:51.712945 12178 raft_consensus.cc:686] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:51.713792 11641 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878"
dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:41482
I20260809 15:51:51.714393 11641 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:51.714735 11641 raft_consensus.cc:3037] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:51.713831 12178 consensus_queue.cc:237] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:51.721381 11641 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:51.724592 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:51.725165 10848 raft_consensus.cc:3032] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:51.725467 10848 raft_consensus.cc:729] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:51.726210 10848 consensus_queue.cc:260] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:51.727319 10848 raft_consensus.cc:3037] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:51.734951 10734 catalog_manager.cc:5495] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 1 to 2, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:51.740398 10848 raft_consensus.cc:2454] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:51:51.743984 11641 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:51.745468 11606 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:51:51.746328 12183 raft_consensus.cc:2781] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:51.746845 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:51.747200 12183 raft_consensus.cc:686] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:51:51.747525 10923 raft_consensus.cc:3037] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:51.748497 12183 consensus_queue.cc:237] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:51.755102 10923 raft_consensus.cc:2454] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:51:51.768720 10734 catalog_manager.cc:5495] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 1 to 2, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 2 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:51.926553 12170 raft_consensus.cc:979] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:51:51.927030 12050 raft_consensus.cc:1067] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:51:51.928637 10843 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45578
I20260809 15:51:51.929138 10843 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:51.929445 10843 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:51.937808 10843 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:51.939453 10843 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:51.940081 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:51.940653 10927 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:51.944553 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:51.945083 11018 raft_consensus.cc:3032] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:51.945344 11018 raft_consensus.cc:729] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:51.946178 11018 consensus_queue.cc:260] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:51.946694 10927 raft_consensus.cc:2454] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:51:51.947633 11018 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:51.947752 10811 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:51.948565 12101 raft_consensus.cc:2781] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:51.949075 12101 raft_consensus.cc:686] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:51.950492 12101 consensus_queue.cc:237] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:51.953967 11018 raft_consensus.cc:2454] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:51:51.964551 10734 catalog_manager.cc:5495] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 1 to 2, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:52.045832 12177 raft_consensus.cc:979] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:51:52.046530 12177 raft_consensus.cc:1067] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:51:52.048635 10843 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45594
I20260809 15:51:52.049062 10843 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:52.049360 10843 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:52.055599 10843 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:52.057229 10843 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:52.058125 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:52.058713 11018 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:52.058132 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:52.059820 10927 raft_consensus.cc:3032] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:52.060492 10927 raft_consensus.cc:729] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:52.061317 10927 consensus_queue.cc:260] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:52.062556 10927 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:52.065356 11018 raft_consensus.cc:2454] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:51:52.066416 10809 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:52.067173 12101 raft_consensus.cc:2781] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:52.068499 12101 raft_consensus.cc:686] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:52.068801 10927 raft_consensus.cc:2454] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:51:52.069612 12101 consensus_queue.cc:237] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:52.084661 10734 catalog_manager.cc:5495] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 1 to 2, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:52.117872 10848 raft_consensus.cc:1261] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:52.119087 12183 consensus_queue.cc:1035] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:52.126638 10927 raft_consensus.cc:1261] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:52.128233 12183 consensus_queue.cc:1035] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:52.156287 10840 raft_consensus.cc:1261] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:52.164911 11018 raft_consensus.cc:1261] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:52.167142 12177 consensus_queue.cc:1035] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:52.172735 12090 consensus_queue.cc:1035] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:52.225358 10927 raft_consensus.cc:1261] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:52.226882 12090 consensus_queue.cc:1035] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:52.307624 10848 raft_consensus.cc:1261] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:52.364358 12199 consensus_queue.cc:1035] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260809 15:51:52.519919 10924 raft_consensus.cc:1261] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260809 15:51:52.531428 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 62748.1us
I20260809 15:51:52.535279 12176 consensus_queue.cc:1035] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
I20260809 15:51:52.542738 10927 raft_consensus.cc:1261] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:52.710227 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
W20260809 15:51:52.749004 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 165506us
I20260809 15:51:52.919772 11010 raft_consensus.cc:1261] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260809 15:51:52.985451 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 70008.4us
W20260809 15:51:53.046703 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 353289us
W20260809 15:51:53.065544 10811 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.528s user 0.019s sys 0.021s
W20260809 15:51:53.099463 10811 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.562s user 0.023s sys 0.021s
W20260809 15:51:53.160194 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 622989us
I20260809 15:51:53.323462 11018 raft_consensus.cc:1261] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260809 15:51:53.343575 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 354665us
W20260809 15:51:53.357036 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 87529.4us
I20260809 15:51:53.378464 12220 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
W20260809 15:51:53.438232 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 95498.7us
W20260809 15:51:53.441075 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 52380.1us
W20260809 15:51:53.442867 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 49260.5us
W20260809 15:51:53.457415 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 49456.8us
I20260809 15:51:53.441534 12245 consensus_queue.cc:1035] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
W20260809 15:51:53.511415 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 68796.8us
I20260809 15:51:53.521777 12219 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:51:53.523030 12219 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:53.530364 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 78657.2us
I20260809 15:51:53.536909 12245 consensus_queue.cc:1035] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:53.459208 12220 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:53.556661 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:53.559115 10924 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:51:53.560248 10924 raft_consensus.cc:595] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Received request to transfer leadership to TS c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:53.568218 12246 consensus_queue.cc:1035] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:53.585074 12226 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:51:53.586119 12226 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:53.549337 12220 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:51:53.583443 12219 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:53.605388 12242 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:51:53.606436 12242 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:53.613477 12242 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:53.616573 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:53.620319 11018 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:51:53.622130 11018 raft_consensus.cc:595] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
W20260809 15:51:53.637404 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 214744us
I20260809 15:51:53.641453 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:53.644277 12226 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:53.645861 12222 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:51:53.646894 12222 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:53.652072 12222 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:53.652429 12234 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:51:53.657390 12227 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:51:53.658450 12227 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:53.665299 12220 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:51:53.666613 12227 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:53.666392 12220 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:53.671414 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:53.675470 12220 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:53.680703 12227 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:51:53.681870 12227 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:53.687704 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:53.688609 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:53.689041 12246 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:51:53.689253 11010 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:51:53.687460 12176 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:51:53.691602 12245 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:51:53.691138 12222 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:51:53.656603 12234 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:53.694501 12222 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:53.690124 12246 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:53.692617 12245 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:53.692026 12176 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:53.705684 12245 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:53.708729 12246 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:53.711164 12176 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:53.715912 10891 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:53.717051 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:53.718494 10927 raft_consensus.cc:2454] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 1.
I20260809 15:51:53.720001 12222 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:53.721174 12220 raft_consensus.cc:2781] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:51:53.722833 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:53.723649 12220 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:51:53.725447 12228 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:51:53.726359 12228 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:53.724417 12220 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:53.729485 11017 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:51:53.730522 11017 raft_consensus.cc:595] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:53.732556 12169 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:51:53.733371 12169 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:53.735153 12228 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:53.736382 10848 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:51:53.737673 10848 raft_consensus.cc:595] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:53.738384 12169 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:53.743875 12224 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:51:53.744505 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:51:53.744846 10848 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:51:53.746464 10848 raft_consensus.cc:595] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:53.744807 12224 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:53.748623 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260809 15:51:53.752077 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:51:53.753911 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:53.756445 12246 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:51:53.757551 12246 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:53.762660 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:51:53.763813 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 78951.6us
I20260809 15:51:53.765174 12227 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:53.768419 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:53.768396 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:53.771071 12224 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:53.773340 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:53.773939 12234 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:51:53.793484 12101 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:51:53.793874 12246 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:51:53.794570 12101 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:53.798890 12101 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:53.800717 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:53.806249 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:53.809662 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:53.804785 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:53.829617 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:53.834225 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:53.837239 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:51:53.833531 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:53.859107 12245 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:51:53.860394 12245 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:53.866312 12245 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:51:53.875633 11606 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:53.878801 12242 raft_consensus.cc:2726] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:51:53.882673 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:53.877517 10848 consensus_queue.cc:237] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:53.884570 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:53.895573 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:53.896626 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:53.862528 12220 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:53.904351 10930 raft_consensus.cc:1226] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 1. Current term is 2. Ops: []
W20260809 15:51:53.916792 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 46736.3us
I20260809 15:51:53.920701 12228 consensus_queue.cc:1046] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:53.921422 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:53.923758 12101 raft_consensus.cc:3032] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:53.924365 12101 raft_consensus.cc:729] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:53.926267 12224 raft_consensus.cc:979] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:51:53.927398 12224 raft_consensus.cc:1067] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
W20260809 15:51:53.927057 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 62224us
I20260809 15:51:53.928112 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:53.932230 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:53.925961 12101 consensus_queue.cc:260] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:53.933750 12101 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:53.942312 10923 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54224
I20260809 15:51:53.944038 10923 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:53.944669 10923 raft_consensus.cc:3037] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:53.919787 11606 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:53.934373 10893 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:53.952137 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:53.961238 12230 raft_consensus.cc:2726] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:51:53.963176 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:53.965948 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:53.965543 10893 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:53.983181 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:51:53.988034 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:53.990775 12250 raft_consensus.cc:2726] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:51:53.986277 12220 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:53.991988 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:53.992491 12225 raft_consensus.cc:2781] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:51:53.993320 12225 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:51:53.994009 12225 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:53.996608 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:51:54.016647 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:54.054834 12169 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:51:54.055483 12210 raft_consensus.cc:979] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:51:54.056478 12210 raft_consensus.cc:1067] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:51:54.055968 12169 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:54.070659 10978 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:54.082867 10978 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:54.084100 10923 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:54.102214 12216 raft_consensus.cc:2726] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:51:54.103020 12214 raft_consensus.cc:979] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3: : Instructing follower c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:51:54.103899 12214 raft_consensus.cc:1067] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Signalling peer c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:51:54.093003 12230 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:51:54.114311 12230 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:54.129451 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 46707.8us
I20260809 15:51:54.147311 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:54.149495 10923 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:54.150812 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:54.151336 11018 raft_consensus.cc:3037] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:54.153204 11641 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb"
dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:41488
I20260809 15:51:54.153692 11641 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:54.144413 12210 raft_consensus.cc:2726] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 15:51:54.154996 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 118574us
I20260809 15:51:54.154021 11641 raft_consensus.cc:3037] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:54.142033 12230 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:54.172024 11018 raft_consensus.cc:2454] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:51:54.173020 10891 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:54.185519 10809 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:54.190650 12230 raft_consensus.cc:2781] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:54.191169 12230 raft_consensus.cc:686] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:54.191999 12230 consensus_queue.cc:237] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:54.195956 10978 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:54.196632 12222 raft_consensus.cc:2726] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:51:54.196753 12169 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:54.197255 12225 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:54.197773 10978 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:54.199206 12222 raft_consensus.cc:2726] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:51:54.205210 12224 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:51:54.205727 12224 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:54.206223 10809 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:54.207316 10809 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:54.207634 12224 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:54.208112 12224 raft_consensus.cc:2726] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:51:54.208611 12224 raft_consensus.cc:2726] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:51:54.208972 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:54.209193 12208 raft_consensus.cc:2726] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:51:54.210892 10809 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:54.211257 12225 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:54.211850 12169 raft_consensus.cc:979] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
W20260809 15:51:54.212090 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 86916.5us
I20260809 15:51:54.212311 12169 raft_consensus.cc:1067] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:51:54.213498 12240 raft_consensus.cc:2726] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:51:54.215272 10809 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:51:54.215937 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 119234us
I20260809 15:51:54.217607 10811 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:54.221737 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:54.222174 10924 raft_consensus.cc:1261] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:54.222425 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:54.224655 11606 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:54.227574 12225 raft_consensus.cc:2726] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 15:51:54.230788 10931 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54236
I20260809 15:51:54.231320 10931 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:54.231395 12101 raft_consensus.cc:2726] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:51:54.231642 10931 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:54.232883 10809 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:54.233741 12258 raft_consensus.cc:2726] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:51:54.235584 12101 raft_consensus.cc:2726] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:51:54.235913 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:54.236948 10811 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:54.237697 12101 raft_consensus.cc:2726] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 15:51:54.238870 10811 consensus_peers.cc:487] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8777c94cbe134bf3aa222691d76ba470. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:54.240468 12245 consensus_queue.cc:1035] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:54.243881 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:54.186393 11641 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:54.247592 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:54.247911 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:54.248008 11641 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:54.247540 10931 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:54.248338 10843 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:51:54.248428 10848 raft_consensus.cc:3037] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:54.250330 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:54.250849 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:54.252609 10923 raft_consensus.cc:3032] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:54.253054 10923 raft_consensus.cc:729] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:54.254061 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:54.254980 10848 raft_consensus.cc:2454] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:51:54.255960 11606 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:51:54.256635 12225 raft_consensus.cc:2781] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:54.254200 10923 consensus_queue.cc:260] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:54.257097 12225 raft_consensus.cc:686] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:51:54.258260 10923 raft_consensus.cc:3037] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:54.257941 12225 consensus_queue.cc:237] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:54.268136 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:51:54.267202 10931 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:51:54.268811 11641 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:54.271981 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:54.272519 11010 raft_consensus.cc:3032] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:54.272792 11010 raft_consensus.cc:729] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:54.273536 11010 consensus_queue.cc:260] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:54.274801 11010 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:54.282627 11641 raft_consensus.cc:2454] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:51:54.284708 10893 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:51:54.285386 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:54.285562 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:54.285979 10848 raft_consensus.cc:3032] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:54.286378 10848 raft_consensus.cc:729] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:54.286926 12234 raft_consensus.cc:2781] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:54.287437 12234 raft_consensus.cc:686] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:54.287065 10848 consensus_queue.cc:260] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:54.288412 10848 raft_consensus.cc:3037] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:54.288651 12234 consensus_queue.cc:237] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:54.287523 11010 raft_consensus.cc:2454] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:51:54.299490 10848 raft_consensus.cc:2454] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:51:54.302963 10843 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:51:54.303522 10923 raft_consensus.cc:2454] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:51:54.307920 10737 catalog_manager.cc:5495] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 1 to 2, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 2 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:54.308981 11010 raft_consensus.cc:1261] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:54.313443 10733 catalog_manager.cc:5495] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 1 to 2, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:54.315595 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:54.320535 10893 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:54.322167 10893 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:54.323346 12177 raft_consensus.cc:2726] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:51:54.324767 12230 raft_consensus.cc:2726] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:51:54.331653 10893 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:54.332617 10848 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45578
I20260809 15:51:54.333292 10848 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:54.333688 10848 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:54.333793 12179 raft_consensus.cc:2726] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:51:54.340618 10893 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:54.342432 12179 raft_consensus.cc:2781] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:54.341625 12240 raft_consensus.cc:2932] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:54.342939 12179 raft_consensus.cc:686] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:54.345619 10923 raft_consensus.cc:2932] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:54.348083 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:51:54.350857 10848 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:54.353212 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:54.353739 10931 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:54.354781 10848 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:54.362649 12179 consensus_queue.cc:237] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:54.365795 10843 raft_consensus.cc:1226] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 1. Current term is 2. Ops: []
I20260809 15:51:54.374963 10931 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:51:54.376060 10811 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:54.376941 12246 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:54.377703 12246 raft_consensus.cc:686] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:54.379176 12246 consensus_queue.cc:237] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:54.382972 11641 raft_consensus.cc:1226] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 1. Current term is 2. Ops: []
I20260809 15:51:54.386564 12198 consensus_queue.cc:1046] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:51:54.388139 12198 raft_consensus.cc:3032] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:54.388700 12198 raft_consensus.cc:729] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:54.397420 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:54.398190 11017 raft_consensus.cc:3032] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:54.398468 11017 raft_consensus.cc:729] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:54.401108 10978 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:51:54.389941 12198 consensus_queue.cc:260] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:54.406704 12198 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:54.404278 11017 consensus_queue.cc:260] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:54.410565 12050 consensus_queue.cc:1046] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:54.417322 11017 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:54.403402 12170 raft_consensus.cc:2726] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:51:54.424188 12101 consensus_queue.cc:1035] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:54.424562 10809 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:54.429903 12259 raft_consensus.cc:2726] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:51:54.461021 11017 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:51:54.492028 10931 consensus_queue.cc:237] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:54.494547 11017 raft_consensus.cc:2932] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
W20260809 15:51:54.514017 10893 consensus_peers.cc:487] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0f891fe62df749108fa553cb0661a874. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:54.514353 11018 raft_consensus.cc:1261] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:54.516836 12178 consensus_queue.cc:1035] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:54.521625 10847 raft_consensus.cc:1261] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:51:54.530242 12179 consensus_queue.cc:1035] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:54.562767 12178 raft_consensus.cc:2932] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:54.566321 10847 raft_consensus.cc:2932] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:54.572185 11018 raft_consensus.cc:2932] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:54.592922 10734 catalog_manager.cc:5495] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:54.604578 10848 consensus_queue.cc:237] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
W20260809 15:51:54.618889 10811 consensus_peers.cc:487] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8feeb57bf5cf4fd1bb17ab7f575222d0. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:51:54.619629 11017 raft_consensus.cc:1261] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260809 15:51:54.621057 10923 raft_consensus.cc:1261] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260809 15:51:54.624459 12246 consensus_queue.cc:1035] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:54.625058 11017 raft_consensus.cc:1261] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:54.624788 10736 catalog_manager.cc:5495] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 1 to 2, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:54.627029 12178 consensus_queue.cc:1035] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:54.627172 12246 consensus_queue.cc:1035] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:54.643360 12240 raft_consensus.cc:2932] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:54.647034 10931 raft_consensus.cc:1261] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:54.649842 12225 consensus_queue.cc:1035] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:54.660130 10930 raft_consensus.cc:2932] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:54.643700 10734 catalog_manager.cc:5495] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 1 to 2, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:54.666044 11012 raft_consensus.cc:2932] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:54.666180 10734 catalog_manager.cc:5495] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 1 to 2, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:54.668712 10734 catalog_manager.cc:5495] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:54.684628 10848 raft_consensus.cc:1261] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:54.689833 10847 raft_consensus.cc:1261] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:54.690121 12225 consensus_queue.cc:1035] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:54.691215 12202 consensus_queue.cc:1035] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260809 15:51:54.720476 10789 auto_rebalancer.cc:254] failed to send replica move request: Illegal state: Leader has not yet committed an operation in its own term
I20260809 15:51:54.734400 10736 catalog_manager.cc:5495] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:51:54.755105 11637 raft_consensus.cc:1261] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:54.762833 12178 consensus_queue.cc:1035] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260809 15:51:54.796828 10789 auto_rebalancer.cc:663] Could not move replica: Incomplete: tablet 5e8d12b5ef9e415cbcb25e62ab228546, TS 20730d75ad954a82b126148f8152a9f3 -> TS c7aa762318cb4e97875333f10679a9f9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:54.797180 10789 auto_rebalancer.cc:264] scheduled replica move failed to complete: Incomplete: tablet 5e8d12b5ef9e415cbcb25e62ab228546, TS 20730d75ad954a82b126148f8152a9f3 -> TS c7aa762318cb4e97875333f10679a9f9 move failed, destination replica disappeared from tablet's Raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:54.800277 11012 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:54.825142 12287 ts_tablet_manager.cc:927] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615)
I20260809 15:51:54.828806 11017 raft_consensus.cc:1261] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:54.830211 12178 consensus_queue.cc:1035] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:54.860873 11012 raft_consensus.cc:1261] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:54.886811 12234 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:54.888942 12287 tablet_copy_client.cc:316] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.1:46615
I20260809 15:51:54.890403 10858 tablet_copy_service.cc:140] P a53441872c0c416b8e2f26f52399f536: Received BeginTabletCopySession request for tablet 8777c94cbe134bf3aa222691d76ba470 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:54342)
I20260809 15:51:54.890959 10858 tablet_copy_service.cc:161] P a53441872c0c416b8e2f26f52399f536: Beginning new tablet copy session on tablet 8777c94cbe134bf3aa222691d76ba470 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:54342: session id = c7aa762318cb4e97875333f10679a9f9-8777c94cbe134bf3aa222691d76ba470
I20260809 15:51:54.910261 10858 tablet_copy_source_session.cc:205] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:54.915179 10930 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:54.916391 12287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8777c94cbe134bf3aa222691d76ba470. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:54.929719 10848 raft_consensus.cc:1261] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:54.919109 12240 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260809 15:51:54.935281 12234 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:54.943792 11017 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:54.945345 12258 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:54.961999 12294 ts_tablet_manager.cc:927] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:55.002118 12294 tablet_copy_client.cc:316] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.2:32783
I20260809 15:51:55.017910 10942 tablet_copy_service.cc:140] P 20730d75ad954a82b126148f8152a9f3: Received BeginTabletCopySession request for tablet 0f891fe62df749108fa553cb0661a874 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:52758)
I20260809 15:51:55.018510 10942 tablet_copy_service.cc:161] P 20730d75ad954a82b126148f8152a9f3: Beginning new tablet copy session on tablet 0f891fe62df749108fa553cb0661a874 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:52758: session id = c7aa762318cb4e97875333f10679a9f9-0f891fe62df749108fa553cb0661a874
I20260809 15:51:55.034175 12287 tablet_copy_client.cc:791] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:55.039180 10942 tablet_copy_source_session.cc:205] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:55.035024 12287 tablet_copy_client.cc:655] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:55.055081 12294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f891fe62df749108fa553cb0661a874. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:55.057869 12287 tablet_copy_client.cc:531] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:55.067818 12287 tablet_bootstrap.cc:492] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:55.080904 12301 ts_tablet_manager.cc:927] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615)
I20260809 15:51:55.082695 12301 tablet_copy_client.cc:316] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.1:46615
I20260809 15:51:55.084153 10858 tablet_copy_service.cc:140] P a53441872c0c416b8e2f26f52399f536: Received BeginTabletCopySession request for tablet 8feeb57bf5cf4fd1bb17ab7f575222d0 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:54342)
I20260809 15:51:55.084642 10858 tablet_copy_service.cc:161] P a53441872c0c416b8e2f26f52399f536: Beginning new tablet copy session on tablet 8feeb57bf5cf4fd1bb17ab7f575222d0 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:54342: session id = c7aa762318cb4e97875333f10679a9f9-8feeb57bf5cf4fd1bb17ab7f575222d0
I20260809 15:51:55.092207 12294 tablet_copy_client.cc:791] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:55.093003 10858 tablet_copy_source_session.cc:205] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:51:55.093437 12294 tablet_copy_client.cc:655] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:55.099187 12301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8feeb57bf5cf4fd1bb17ab7f575222d0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:51:55.102578 12294 tablet_copy_client.cc:531] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:55.128268 12301 tablet_copy_client.cc:791] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:51:55.132870 12301 tablet_copy_client.cc:655] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:51:55.151940 12301 tablet_copy_client.cc:531] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:51:55.156701 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:55.174132 12287 tablet_bootstrap.cc:492] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:55.175418 12287 tablet_bootstrap.cc:492] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:55.176446 12287 ts_tablet_manager.cc:1397] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.109s user 0.025s sys 0.005s
I20260809 15:51:55.187065 12294 tablet_bootstrap.cc:492] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:55.181864 12287 raft_consensus.cc:348] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:55.188838 12287 raft_consensus.cc:729] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:55.195631 12287 consensus_queue.cc:260] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:55.202737 12287 ts_tablet_manager.cc:1428] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.026s user 0.003s sys 0.007s
I20260809 15:51:55.207098 10858 tablet_copy_service.cc:342] P a53441872c0c416b8e2f26f52399f536: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-8777c94cbe134bf3aa222691d76ba470 received from {username='slave'} at 127.0.0.1:54342
I20260809 15:51:55.207674 10858 tablet_copy_service.cc:433] P a53441872c0c416b8e2f26f52399f536: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-8777c94cbe134bf3aa222691d76ba470 on tablet 8777c94cbe134bf3aa222691d76ba470 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:55.236959 12301 tablet_bootstrap.cc:492] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:51:55.309219 11641 raft_consensus.cc:1203] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:55.351626 12301 tablet_bootstrap.cc:492] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:55.352604 12301 tablet_bootstrap.cc:492] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:55.353266 12301 ts_tablet_manager.cc:1397] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.117s user 0.014s sys 0.014s
I20260809 15:51:55.367527 10931 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:51:55.368252 10931 raft_consensus.cc:595] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:55.371685 12301 raft_consensus.cc:348] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:55.372599 12301 raft_consensus.cc:729] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:55.373492 11017 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:51:55.374293 11017 raft_consensus.cc:595] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:51:55.373296 12301 consensus_queue.cc:260] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:55.376832 11017 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:51:55.377413 11017 raft_consensus.cc:595] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:55.384963 12301 ts_tablet_manager.cc:1428] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.031s user 0.008s sys 0.001s
I20260809 15:51:55.386953 10844 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:51:55.387504 10844 raft_consensus.cc:595] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:55.387813 10858 tablet_copy_service.cc:342] P a53441872c0c416b8e2f26f52399f536: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-8feeb57bf5cf4fd1bb17ab7f575222d0 received from {username='slave'} at 127.0.0.1:54342
I20260809 15:51:55.388222 10858 tablet_copy_service.cc:433] P a53441872c0c416b8e2f26f52399f536: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-8feeb57bf5cf4fd1bb17ab7f575222d0 on tablet 8feeb57bf5cf4fd1bb17ab7f575222d0 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:55.391844 10923 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:51:55.392397 10923 raft_consensus.cc:595] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:51:55.400053 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 5
I20260809 15:51:55.400545 12170 raft_consensus.cc:979] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:51:55.400972 12170 raft_consensus.cc:1067] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:51:55.403414 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:51:55.404651 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:55.415031 10847 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45578
I20260809 15:51:55.415560 10847 raft_consensus.cc:482] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:55.415917 10847 raft_consensus.cc:3037] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:55.424054 10847 raft_consensus.cc:504] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:55.426919 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:51:55.427582 11637 raft_consensus.cc:3037] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:55.427914 10847 leader_election.cc:288] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:51:55.429325 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:55.429932 11012 raft_consensus.cc:3032] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:55.430256 11012 raft_consensus.cc:729] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:55.427752 12294 tablet_bootstrap.cc:492] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:51:55.430927 11012 consensus_queue.cc:260] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:55.431943 12294 tablet_bootstrap.cc:492] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:51:55.432070 11012 raft_consensus.cc:3037] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:55.432637 12294 ts_tablet_manager.cc:1397] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.246s user 0.015s sys 0.031s
I20260809 15:51:55.440570 11012 raft_consensus.cc:2454] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:51:55.441720 10809 leader_election.cc:302] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:55.442781 12207 raft_consensus.cc:2781] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:55.443938 12207 raft_consensus.cc:686] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:55.444963 12207 consensus_queue.cc:237] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:55.447957 12294 raft_consensus.cc:348] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:55.448896 12294 raft_consensus.cc:729] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:51:55.455622 11637 raft_consensus.cc:2454] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:51:55.458562 12294 consensus_queue.cc:260] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:51:55.480770 10734 catalog_manager.cc:5495] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 1 to 2, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:55.482534 12294 ts_tablet_manager.cc:1428] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.050s user 0.004s sys 0.004s
I20260809 15:51:55.493507 10942 tablet_copy_service.cc:342] P 20730d75ad954a82b126148f8152a9f3: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-0f891fe62df749108fa553cb0661a874 received from {username='slave'} at 127.0.0.1:52758
I20260809 15:51:55.493974 10942 tablet_copy_service.cc:433] P 20730d75ad954a82b126148f8152a9f3: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-0f891fe62df749108fa553cb0661a874 on tablet 0f891fe62df749108fa553cb0661a874 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:55.557955 11637 raft_consensus.cc:1203] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20260809 15:51:55.607604 12234 raft_consensus.cc:979] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:51:55.608152 12179 raft_consensus.cc:1067] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:51:55.609611 10848 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45594
I20260809 15:51:55.610152 10848 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:55.610502 10848 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:55.616830 10848 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:55.620162 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:55.620679 10923 raft_consensus.cc:3032] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:55.620910 10923 raft_consensus.cc:729] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:55.621544 10923 consensus_queue.cc:260] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:55.622488 10923 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:55.630285 10923 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:51:55.631220 10811 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:55.632006 12246 raft_consensus.cc:2781] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:55.632458 12246 raft_consensus.cc:686] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:55.633292 12246 consensus_queue.cc:237] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:55.637562 10848 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:55.639837 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:55.640348 11017 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:55.651368 11017 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:51:55.658562 10733 catalog_manager.cc:5495] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 1 to 2, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:55.710381 12198 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:51:55.710892 12198 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:55.712785 12198 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:51:55.714535 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:51:55.719084 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:51:55.719797 11641 raft_consensus.cc:2454] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:51:55.721879 10978 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:55.725013 12246 raft_consensus.cc:1050] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:55.727732 12246 consensus_queue.cc:237] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:55.730597 12198 raft_consensus.cc:2781] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:51:55.730983 12198 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:51:55.733286 12170 raft_consensus.cc:979] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:51:55.734282 12198 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:51:55.734472 12222 raft_consensus.cc:1067] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:51:55.736522 10923 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54236
I20260809 15:51:55.737082 10923 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:55.737478 10923 raft_consensus.cc:3037] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:55.741398 12198 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:55.743543 12198 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:51:55.745064 12234 raft_consensus.cc:979] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:51:55.745468 12234 raft_consensus.cc:1067] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:51:55.746811 11017 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50542
I20260809 15:51:55.747298 11017 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:55.747605 11017 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:51:55.747892 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:55.754902 11017 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:55.757766 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:55.758330 10930 raft_consensus.cc:3032] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:51:55.758611 10930 raft_consensus.cc:729] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:55.758970 11017 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:51:55.759287 10930 consensus_queue.cc:260] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:55.760466 10930 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:51:55.765632 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:51:55.766230 11641 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:51:55.766989 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:51:55.767530 11639 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:51:55.774511 11641 raft_consensus.cc:2454] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:51:55.775601 11639 raft_consensus.cc:2454] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:51:55.781258 10930 raft_consensus.cc:2454] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:51:55.784559 10978 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:55.788061 10923 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:55.789991 10923 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:51:55.799260 11012 raft_consensus.cc:1261] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:55.800813 12246 consensus_queue.cc:1035] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:55.801915 11639 raft_consensus.cc:1261] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:55.811928 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:51:55.812647 11639 raft_consensus.cc:3037] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:55.802364 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:55.816133 11012 raft_consensus.cc:3032] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:55.816452 11012 raft_consensus.cc:729] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:55.817181 11012 consensus_queue.cc:260] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:55.802745 10923 raft_consensus.cc:1261] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:55.819865 12207 consensus_queue.cc:1035] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:55.822858 12207 consensus_queue.cc:1035] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:55.826118 11012 raft_consensus.cc:3037] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:55.830425 12198 raft_consensus.cc:2781] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:51:55.831032 12198 raft_consensus.cc:686] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:55.832015 12198 consensus_queue.cc:237] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:51:55.834342 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 49859.6us
I20260809 15:51:55.848570 10978 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:55.850785 12222 raft_consensus.cc:2781] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:51:55.851363 12222 raft_consensus.cc:686] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:55.852285 12222 consensus_queue.cc:237] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:55.863873 10733 catalog_manager.cc:5495] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 2 to 3, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 3 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:55.872148 11639 raft_consensus.cc:2454] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:51:55.873925 12207 raft_consensus.cc:979] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:51:55.878515 12207 raft_consensus.cc:1067] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:51:55.880534 10931 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54224
I20260809 15:51:55.886785 10931 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:55.887264 10931 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:55.885916 11641 raft_consensus.cc:2932] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:55.881902 12258 raft_consensus.cc:2932] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:55.888096 12234 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:51:55.888648 12234 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:55.890769 12234 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:51:55.892196 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:51:55.893036 11637 raft_consensus.cc:2354] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 2. Current term is 3.
I20260809 15:51:55.894167 10893 leader_election.cc:398] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Vote denied by peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685) with higher term. Message: Invalid argument: T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 2. Current term is 3.
I20260809 15:51:55.894490 10893 leader_election.cc:401] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260809 15:51:55.895180 12234 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 3
I20260809 15:51:55.898355 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:51:55.893004 11012 raft_consensus.cc:2454] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:51:55.903074 10891 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:55.905565 10733 catalog_manager.cc:5495] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:55.912828 12234 raft_consensus.cc:2726] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685) with higher term. Message: Invalid argument: T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 2. Current term is 3.
I20260809 15:51:55.928480 11637 raft_consensus.cc:1203] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:51:55.931990 11017 raft_consensus.cc:2932] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:55.945806 10931 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:55.957224 10931 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:55.957983 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:55.959022 11012 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:55.951450 12319 raft_consensus.cc:2781] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:55.976620 12319 raft_consensus.cc:686] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:55.980029 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:55.980681 10848 raft_consensus.cc:3032] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:55.981010 10848 raft_consensus.cc:729] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:55.984246 11012 raft_consensus.cc:2454] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:51:55.985188 10891 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:55.992779 10848 consensus_queue.cc:260] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:55.999869 10848 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:56.009096 10848 raft_consensus.cc:2454] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:51:55.977708 12319 consensus_queue.cc:237] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:56.001670 10733 catalog_manager.cc:5495] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260809 15:51:56.034448 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 45738.9us
I20260809 15:51:56.003405 12321 raft_consensus.cc:2781] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:56.039950 11012 raft_consensus.cc:1261] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:56.044831 12326 consensus_queue.cc:1035] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:51:56.048897 10931 raft_consensus.cc:2932] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:56.051709 12321 raft_consensus.cc:686] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:56.057096 12321 consensus_queue.cc:237] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:56.062671 10930 raft_consensus.cc:1261] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:56.074424 12240 consensus_queue.cc:1035] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
W20260809 15:51:56.094856 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 106726us
I20260809 15:51:56.101709 10848 consensus_queue.cc:237] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:56.103978 11641 raft_consensus.cc:1261] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:56.105916 12323 consensus_queue.cc:1035] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:56.111268 10736 catalog_manager.cc:5495] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 1 to 2, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:56.136322 11637 raft_consensus.cc:1261] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:56.139550 12326 consensus_queue.cc:1035] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:51:56.149039 12326 raft_consensus.cc:2932] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:56.156958 11012 raft_consensus.cc:1261] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:56.156538 11637 raft_consensus.cc:2932] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:56.171020 10733 catalog_manager.cc:5495] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 1 to 2, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:56.202551 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 8777c94cbe134bf3aa222691d76ba470 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:56.203344 12323 consensus_queue.cc:1035] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:56.209805 10734 catalog_manager.cc:5495] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:56.216787 11017 raft_consensus.cc:1261] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:56.219334 11018 raft_consensus.cc:2932] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:56.223487 10924 raft_consensus.cc:1261] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:51:56.226320 12328 consensus_queue.cc:1035] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:51:56.229511 10913 tablet_service.cc:1514] Processing DeleteTablet for tablet 8777c94cbe134bf3aa222691d76ba470 with delete_type TABLET_DATA_TOMBSTONED (TS 20730d75ad954a82b126148f8152a9f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54216
I20260809 15:51:56.238093 12207 consensus_queue.cc:1035] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.002s
W20260809 15:51:56.244096 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 53653.6us
I20260809 15:51:56.255901 11637 raft_consensus.cc:1261] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:51:56.329452 12340 tablet_replica.cc:330] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3: stopping tablet replica
I20260809 15:51:56.332612 12340 raft_consensus.cc:2229] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:56.334542 12340 raft_consensus.cc:2258] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:56.294596 12322 consensus_queue.cc:1035] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260809 15:51:56.360015 11641 raft_consensus.cc:1261] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:51:56.397210 12340 ts_tablet_manager.cc:1905] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260809 15:51:56.498468 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 99375.2us
W20260809 15:51:56.502171 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 130473us
I20260809 15:51:56.398877 12346 consensus_queue.cc:1035] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.122s
W20260809 15:51:56.501498 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 129001us
I20260809 15:51:56.521687 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:56.526692 11010 raft_consensus.cc:1261] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:56.544258 11641 raft_consensus.cc:1261] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:56.542058 12234 consensus_queue.cc:1035] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.024s
W20260809 15:51:56.612411 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 230711us
I20260809 15:51:56.648669 12234 raft_consensus.cc:1050] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:56.651981 12234 consensus_queue.cc:237] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:56.663153 12336 raft_consensus.cc:1050] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:51:56.671572 12327 consensus_queue.cc:1035] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260809 15:51:56.679667 10848 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:51:56.680270 10848 raft_consensus.cc:595] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:56.685102 10930 raft_consensus.cc:1261] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:51:56.684392 12336 consensus_queue.cc:237] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:56.696087 12316 consensus_queue.cc:1035] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:51:56.726929 11012 raft_consensus.cc:1261] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:56.736284 10923 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:51:56.736982 10923 raft_consensus.cc:595] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:56.781270 12340 ts_tablet_manager.cc:1918] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:56.781773 12340 log.cc:1196] T 8777c94cbe134bf3aa222691d76ba470 P 20730d75ad954a82b126148f8152a9f3: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-1-root/wals/8777c94cbe134bf3aa222691d76ba470
I20260809 15:51:56.782581 10848 raft_consensus.cc:1261] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:56.783593 10722 catalog_manager.cc:4841] TS 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): tablet 8777c94cbe134bf3aa222691d76ba470 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:56.798583 12321 consensus_queue.cc:1035] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:51:56.804553 11018 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:51:56.805353 11018 raft_consensus.cc:595] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:51:56.814448 12323 raft_consensus.cc:979] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:51:56.815259 12323 raft_consensus.cc:1067] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
W20260809 15:51:56.817006 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 62449.3us
W20260809 15:51:56.819955 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 59841.2us
I20260809 15:51:56.823796 11642 raft_consensus.cc:1261] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:51:56.839438 11011 raft_consensus.cc:1261] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:56.840638 12353 consensus_queue.cc:1035] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:51:56.844870 10931 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:51:56.845422 10931 raft_consensus.cc:595] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:51:56.846643 10931 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54224
I20260809 15:51:56.847067 10931 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:56.847375 10931 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:56.849124 10923 raft_consensus.cc:1261] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:51:56.861130 12319 raft_consensus.cc:979] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:51:56.863336 12318 raft_consensus.cc:1067] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:51:56.864290 12198 raft_consensus.cc:979] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:51:56.864806 12198 raft_consensus.cc:1067] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:51:56.870333 10848 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:51:56.870937 10848 raft_consensus.cc:595] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:56.881430 10931 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:56.887974 10931 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:56.890138 10848 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45578
I20260809 15:51:56.891497 10848 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:56.891963 10848 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:56.882301 12178 consensus_queue.cc:1035] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.015s
I20260809 15:51:56.899710 11018 raft_consensus.cc:1261] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:51:56.904990 10848 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:56.907264 10848 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:56.908285 12353 consensus_queue.cc:1035] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:51:56.941635 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:56.989795 11018 raft_consensus.cc:3032] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:56.990302 11018 raft_consensus.cc:729] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:56.991067 11018 consensus_queue.cc:260] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:56.992506 11018 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
W20260809 15:51:57.006800 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 83762.9us
W20260809 15:51:57.023262 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 156886us
I20260809 15:51:57.035455 11011 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50542
I20260809 15:51:57.036059 11011 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:57.036985 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:57.037652 11010 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:57.036425 11011 raft_consensus.cc:3037] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:57.042586 11018 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:51:57.044130 10809 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:57.045162 12258 raft_consensus.cc:2781] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:57.049675 11010 raft_consensus.cc:2454] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:51:57.049140 12246 consensus_queue.cc:1035] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:51:57.045915 12258 raft_consensus.cc:686] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:57.056838 10931 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:51:57.057487 10931 raft_consensus.cc:595] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Received request to transfer leadership to TS c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:57.058267 12258 consensus_queue.cc:237] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:57.063606 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:57.064201 10931 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:57.056852 11011 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:57.075728 10931 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
W20260809 15:51:57.076601 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 206081us
I20260809 15:51:57.082401 11011 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:57.089419 12319 consensus_queue.cc:1035] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:57.093580 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:57.094237 10848 raft_consensus.cc:3032] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:57.094512 10848 raft_consensus.cc:729] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:57.095646 11642 raft_consensus.cc:1261] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:57.095216 10848 consensus_queue.cc:260] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:57.096798 10848 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:57.098865 10840 raft_consensus.cc:1261] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:51:57.126387 10848 raft_consensus.cc:2454] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:51:57.136422 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:57.138267 10848 raft_consensus.cc:3037] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:57.149248 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260809 15:51:57.149709 12353 raft_consensus.cc:979] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:51:57.150385 12353 raft_consensus.cc:1067] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:51:57.151140 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:51:57.151837 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:57.158557 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:57.159226 10930 raft_consensus.cc:3032] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:57.159524 10930 raft_consensus.cc:729] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:57.160341 10930 consensus_queue.cc:260] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:57.161586 10930 raft_consensus.cc:3037] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:57.168267 10848 raft_consensus.cc:2454] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:51:57.174257 10978 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:57.175915 12316 raft_consensus.cc:2781] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:57.176652 12316 raft_consensus.cc:686] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:57.177829 12316 consensus_queue.cc:237] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:57.189092 12369 consensus_queue.cc:1035] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:57.192247 12369 raft_consensus.cc:979] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3: : Instructing follower c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:51:57.191279 12326 raft_consensus.cc:2932] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:57.192945 12369 raft_consensus.cc:1067] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Signalling peer c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:51:57.195560 11012 raft_consensus.cc:2932] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:57.196774 10893 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:51:57.197592 12320 consensus_queue.cc:1035] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:57.198760 12369 raft_consensus.cc:2781] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:57.199429 12369 raft_consensus.cc:686] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:57.200382 12369 consensus_queue.cc:237] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:57.206681 11637 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d"
dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:41488
I20260809 15:51:57.211655 11637 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:57.212075 11637 raft_consensus.cc:3037] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:57.242280 10926 raft_consensus.cc:2932] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:57.247046 10924 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54224
I20260809 15:51:57.247717 10924 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:57.248193 10924 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:57.259895 11637 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:57.265295 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:57.264402 11639 raft_consensus.cc:2932] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:57.265974 10843 raft_consensus.cc:3037] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:57.276046 10930 raft_consensus.cc:2454] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:51:57.283354 10843 raft_consensus.cc:2454] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:51:57.309255 10736 catalog_manager.cc:5495] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 1 to 2, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:57.305877 11606 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:51:57.326198 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:57.326848 10926 raft_consensus.cc:3032] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:57.327204 10926 raft_consensus.cc:729] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:57.329838 10926 consensus_queue.cc:260] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:57.331310 10926 raft_consensus.cc:3037] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:57.347828 10926 raft_consensus.cc:2454] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:51:57.324316 12358 raft_consensus.cc:2781] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:57.324966 11637 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:57.371181 12358 raft_consensus.cc:686] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:51:57.383090 12358 consensus_queue.cc:237] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:57.381590 10734 catalog_manager.cc:5495] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:57.409866 10733 catalog_manager.cc:5495] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 1 to 2, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:57.438719 10924 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:57.445919 10926 raft_consensus.cc:1226] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 1. Current term is 2. Ops: []
I20260809 15:51:57.449923 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:57.455533 10924 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:57.456267 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:57.505429 11012 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:57.579952 11012 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
W20260809 15:51:57.605453 12380 log.cc:924] Time spent T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.185s user 0.001s sys 0.000s
W20260809 15:51:57.609436 12377 log.cc:924] Time spent T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.195s user 0.001s sys 0.000s
I20260809 15:51:57.614477 10891 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:51:57.709434 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 75166.5us
I20260809 15:51:57.647748 12371 raft_consensus.cc:2781] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election won for term 2
W20260809 15:51:57.713977 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 82288.8us
W20260809 15:51:57.715009 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 97803.8us
I20260809 15:51:57.726749 10843 raft_consensus.cc:3032] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:57.727164 10843 raft_consensus.cc:729] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:57.727970 10843 consensus_queue.cc:260] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:51:57.732132 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 77808.1us
I20260809 15:51:57.732657 10843 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:57.741781 10843 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
W20260809 15:51:57.742460 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 155939us
W20260809 15:51:57.784389 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 51394.3us
I20260809 15:51:57.807955 10926 raft_consensus.cc:1261] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:57.799176 12359 raft_consensus.cc:2932] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:57.829276 12353 consensus_queue.cc:1035] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:57.842048 11636 raft_consensus.cc:2932] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:57.874519 10926 raft_consensus.cc:1261] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:57.861421 12371 raft_consensus.cc:686] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:57.924377 10931 raft_consensus.cc:1261] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:57.924558 11014 raft_consensus.cc:1261] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:57.927584 10840 raft_consensus.cc:1261] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:57.930022 12390 consensus_queue.cc:1035] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260809 15:51:57.933840 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 64830.7us
I20260809 15:51:57.928462 12170 consensus_queue.cc:1035] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.004s
I20260809 15:51:57.950798 12371 consensus_queue.cc:237] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:57.966214 10848 raft_consensus.cc:1261] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:57.974956 12396 consensus_queue.cc:1035] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.016s
I20260809 15:51:57.986279 12372 raft_consensus.cc:979] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:51:57.973104 11016 raft_consensus.cc:2932] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:57.987118 12372 raft_consensus.cc:1067] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:51:57.991397 12326 consensus_queue.cc:1035] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:57.982426 11017 raft_consensus.cc:1261] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:57.980751 12358 consensus_queue.cc:1035] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
W20260809 15:51:58.034865 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 116697us
I20260809 15:51:58.031530 12179 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.011s
I20260809 15:51:58.053918 10840 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45594
I20260809 15:51:58.054668 10840 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:58.055163 10840 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:51:58.063994 10844 raft_consensus.cc:1261] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:58.106518 12359 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:58.065753 10842 raft_consensus.cc:2932] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:58.144490 10840 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:58.151156 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:58.152294 11016 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:51:58.164909 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:58.165832 10923 raft_consensus.cc:3032] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:51:58.166193 10923 raft_consensus.cc:729] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:58.167055 10923 consensus_queue.cc:260] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:58.168327 10923 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:51:58.168490 10840 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:58.194281 11016 raft_consensus.cc:2454] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:51:58.203958 10809 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:58.210017 10923 raft_consensus.cc:2454] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:51:58.211572 12323 raft_consensus.cc:2781] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:51:58.251137 12323 raft_consensus.cc:686] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:58.390331 12323 consensus_queue.cc:237] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:58.486904 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:58.548705 10737 catalog_manager.cc:5495] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 1 to 2, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 2 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:58.589942 10929 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
W20260809 15:51:58.590859 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 211741us
I20260809 15:51:58.591189 10929 raft_consensus.cc:595] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:58.605759 10733 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33038 (request call id 261) took 1211 ms. Trace:
I20260809 15:51:58.640025 10734 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59504 (request call id 161) took 1286 ms. Trace:
I20260809 15:51:58.640550 10734 rpcz_store.cc:276] 0809 15:51:57.353495 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:57.354163 (+ 668us) service_pool.cc:227] Handling call
0809 15:51:58.639808 (+1285645us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 15:51:57.871581 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 15:51:57.943577 (+ 71996us) write_op.cc:432] Acquiring schema lock in shared mode
0809 15:51:57.943619 (+ 42us) write_op.cc:435] Acquired schema lock
0809 15:51:57.943638 (+ 19us) tablet.cc:660] Decoding operations
0809 15:51:57.951059 (+ 7421us) write_op.cc:620] Acquiring the partition lock for write op
0809 15:51:57.952234 (+ 1175us) write_op.cc:641] Partition lock acquired for write op
0809 15:51:57.952266 (+ 32us) tablet.cc:683] Acquiring locks for 1 operations
0809 15:51:57.952472 (+ 206us) tablet.cc:699] Row locks acquired
0809 15:51:57.952492 (+ 20us) write_op.cc:260] PREPARE: finished
0809 15:51:57.952633 (+ 141us) write_op.cc:270] Start()
0809 15:51:57.952755 (+ 122us) write_op.cc:276] Timestamp: P: 1786290717952588 usec, L: 0
0809 15:51:57.952781 (+ 26us) op_driver.cc:331] REPLICATION: starting
0809 15:51:57.953380 (+ 599us) log.cc:842] Serialized 587 byte log entry
0809 15:51:58.326184 (+372804us) op_driver.cc:444] REPLICATION: finished
0809 15:51:58.444498 (+118314us) write_op.cc:301] APPLY: starting
0809 15:51:58.444566 (+ 68us) tablet.cc:1365] starting BulkCheckPresence
0809 15:51:58.444659 (+ 93us) tablet.cc:1368] finished BulkCheckPresence
0809 15:51:58.444765 (+ 106us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 15:51:58.444936 (+ 171us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 15:51:58.445062 (+ 126us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 15:51:58.445106 (+ 44us) write_op.cc:312] APPLY: finished
0809 15:51:58.445263 (+ 157us) log.cc:842] Serialized 23 byte log entry
0809 15:51:58.445427 (+ 164us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:51:58.445485 (+ 58us) write_op.cc:454] Released schema lock
0809 15:51:58.445519 (+ 34us) write_op.cc:341] FINISH: Updating metrics
0809 15:51:58.445741 (+ 222us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:51:58.445763 (+ 22us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":49378,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":118112,"apply.run_cpu_time_us":1452,"apply.run_wall_time_us":1451,"num_ops":1,"prepare.queue_time_us":468484,"prepare.run_cpu_time_us":2939,"prepare.run_wall_time_us":89074,"replication_time_us":373279,"thread_start_us":87490,"threads_started":1}]]}
I20260809 15:51:58.655007 11012 raft_consensus.cc:1261] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:58.664448 10736 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33054 (request call id 280) took 1366 ms. Trace:
I20260809 15:51:58.664875 10736 rpcz_store.cc:276] 0809 15:51:57.298404 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:57.298573 (+ 169us) service_pool.cc:227] Handling call
0809 15:51:58.664252 (+1365679us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 15:51:57.610512 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 15:51:57.611190 (+ 678us) write_op.cc:432] Acquiring schema lock in shared mode
0809 15:51:57.611241 (+ 51us) write_op.cc:435] Acquired schema lock
0809 15:51:57.611266 (+ 25us) tablet.cc:660] Decoding operations
0809 15:51:57.611676 (+ 410us) write_op.cc:620] Acquiring the partition lock for write op
0809 15:51:57.611748 (+ 72us) write_op.cc:641] Partition lock acquired for write op
0809 15:51:57.611774 (+ 26us) tablet.cc:683] Acquiring locks for 1 operations
0809 15:51:57.612004 (+ 230us) tablet.cc:699] Row locks acquired
0809 15:51:57.612030 (+ 26us) write_op.cc:260] PREPARE: finished
0809 15:51:57.612198 (+ 168us) write_op.cc:270] Start()
0809 15:51:57.612343 (+ 145us) write_op.cc:276] Timestamp: P: 1786290717612153 usec, L: 0
0809 15:51:57.612369 (+ 26us) op_driver.cc:331] REPLICATION: starting
0809 15:51:57.613133 (+ 764us) log.cc:842] Serialized 532 byte log entry
0809 15:51:57.841781 (+228648us) spinlock_profiling.cc:243] Waited 9.14 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f79665ad5fe 00007f796f3e4a39 00007f79665ca8bb 00007f79665c8cbf 00007f79665cae22 00007f79691745bb
0809 15:51:57.957897 (+116116us) spinlock_profiling.cc:243] Waited 3 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:51:58.251365 (+293468us) op_driver.cc:444] REPLICATION: finished
0809 15:51:58.442772 (+191407us) write_op.cc:301] APPLY: starting
0809 15:51:58.442857 (+ 85us) tablet.cc:1365] starting BulkCheckPresence
0809 15:51:58.442953 (+ 96us) tablet.cc:1368] finished BulkCheckPresence
0809 15:51:58.442975 (+ 22us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 15:51:58.443213 (+ 238us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 15:51:58.443345 (+ 132us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 15:51:58.443399 (+ 54us) write_op.cc:312] APPLY: finished
0809 15:51:58.443575 (+ 176us) log.cc:842] Serialized 23 byte log entry
0809 15:51:58.443746 (+ 171us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:51:58.443813 (+ 67us) write_op.cc:454] Released schema lock
0809 15:51:58.443850 (+ 37us) write_op.cc:341] FINISH: Updating metrics
0809 15:51:58.444081 (+ 231us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:51:58.444116 (+ 35us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":48219,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":183513,"apply.run_cpu_time_us":1570,"apply.run_wall_time_us":1571,"num_ops":1,"prepare.queue_time_us":298342,"prepare.run_cpu_time_us":11847,"prepare.run_wall_time_us":248955,"raft.queue_time_us":656958,"raft.run_cpu_time_us":58476,"raft.run_wall_time_us":223835,"replication_time_us":634793,"spinlock_wait_cycles":27925504,"thread_start_us":663512,"threads_started":4,"wal-append.queue_time_us":299675}]]}
I20260809 15:51:58.674121 10847 raft_consensus.cc:1261] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:58.607501 10733 rpcz_store.cc:276] 0809 15:51:57.393672 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:57.393828 (+ 156us) service_pool.cc:227] Handling call
0809 15:51:58.605431 (+1211603us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 15:51:57.953892 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 15:51:57.954487 (+ 595us) write_op.cc:432] Acquiring schema lock in shared mode
0809 15:51:57.954515 (+ 28us) write_op.cc:435] Acquired schema lock
0809 15:51:57.954532 (+ 17us) tablet.cc:660] Decoding operations
0809 15:51:57.955066 (+ 534us) write_op.cc:620] Acquiring the partition lock for write op
0809 15:51:57.955136 (+ 70us) write_op.cc:641] Partition lock acquired for write op
0809 15:51:57.955197 (+ 61us) tablet.cc:683] Acquiring locks for 1 operations
0809 15:51:57.955498 (+ 301us) tablet.cc:699] Row locks acquired
0809 15:51:57.955542 (+ 44us) write_op.cc:260] PREPARE: finished
0809 15:51:57.955757 (+ 215us) write_op.cc:270] Start()
0809 15:51:57.955978 (+ 221us) write_op.cc:276] Timestamp: P: 1786290717955683 usec, L: 0
0809 15:51:57.956020 (+ 42us) op_driver.cc:331] REPLICATION: starting
0809 15:51:57.956976 (+ 956us) log.cc:842] Serialized 532 byte log entry
0809 15:51:58.414468 (+457492us) op_driver.cc:444] REPLICATION: finished
0809 15:51:58.451306 (+ 36838us) write_op.cc:301] APPLY: starting
0809 15:51:58.451381 (+ 75us) tablet.cc:1365] starting BulkCheckPresence
0809 15:51:58.451467 (+ 86us) tablet.cc:1368] finished BulkCheckPresence
0809 15:51:58.451571 (+ 104us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 15:51:58.451752 (+ 181us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 15:51:58.451890 (+ 138us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 15:51:58.451928 (+ 38us) write_op.cc:312] APPLY: finished
0809 15:51:58.452090 (+ 162us) log.cc:842] Serialized 23 byte log entry
0809 15:51:58.452721 (+ 631us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:51:58.452787 (+ 66us) write_op.cc:454] Released schema lock
0809 15:51:58.452819 (+ 32us) write_op.cc:341] FINISH: Updating metrics
0809 15:51:58.453100 (+ 281us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:51:58.453125 (+ 25us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":31393,"apply.run_cpu_time_us":1540,"apply.run_wall_time_us":7194,"num_ops":1,"prepare.queue_time_us":524492,"prepare.run_cpu_time_us":3521,"prepare.run_wall_time_us":4582,"replication_time_us":458230,"thread_start_us":33044,"threads_started":1}]]}
I20260809 15:51:58.689138 10847 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:51:58.691843 10847 raft_consensus.cc:595] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:51:58.709594 10737 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33062 (request call id 263) took 1123 ms. Trace:
W20260809 15:51:58.717937 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 92234.8us
I20260809 15:51:58.710141 10737 rpcz_store.cc:276] 0809 15:51:57.585980 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:51:57.586144 (+ 164us) service_pool.cc:227] Handling call
0809 15:51:58.709529 (+1123385us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 15:51:58.557211 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 15:51:58.558096 (+ 885us) write_op.cc:432] Acquiring schema lock in shared mode
0809 15:51:58.558161 (+ 65us) write_op.cc:435] Acquired schema lock
0809 15:51:58.558202 (+ 41us) tablet.cc:660] Decoding operations
0809 15:51:58.558818 (+ 616us) write_op.cc:620] Acquiring the partition lock for write op
0809 15:51:58.558937 (+ 119us) write_op.cc:641] Partition lock acquired for write op
0809 15:51:58.558997 (+ 60us) tablet.cc:683] Acquiring locks for 1 operations
0809 15:51:58.559455 (+ 458us) tablet.cc:699] Row locks acquired
0809 15:51:58.559499 (+ 44us) write_op.cc:260] PREPARE: finished
0809 15:51:58.559678 (+ 179us) write_op.cc:270] Start()
0809 15:51:58.559870 (+ 192us) write_op.cc:276] Timestamp: P: 1786290718559611 usec, L: 0
0809 15:51:58.559923 (+ 53us) op_driver.cc:331] REPLICATION: starting
0809 15:51:58.561085 (+ 1162us) log.cc:842] Serialized 532 byte log entry
0809 15:51:58.649070 (+ 87985us) op_driver.cc:444] REPLICATION: finished
0809 15:51:58.649876 (+ 806us) write_op.cc:301] APPLY: starting
0809 15:51:58.649991 (+ 115us) tablet.cc:1365] starting BulkCheckPresence
0809 15:51:58.650195 (+ 204us) tablet.cc:1368] finished BulkCheckPresence
0809 15:51:58.650374 (+ 179us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 15:51:58.650746 (+ 372us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 15:51:58.651063 (+ 317us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 15:51:58.651139 (+ 76us) write_op.cc:312] APPLY: finished
0809 15:51:58.678133 (+ 26994us) log.cc:842] Serialized 23 byte log entry
0809 15:51:58.678347 (+ 214us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:51:58.678436 (+ 89us) write_op.cc:454] Released schema lock
0809 15:51:58.678514 (+ 78us) write_op.cc:341] FINISH: Updating metrics
0809 15:51:58.683401 (+ 4887us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:51:58.689504 (+ 6103us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":42968,"reported_tablets":3,"child_traces":[["op",{"apply.queue_time_us":473,"apply.run_cpu_time_us":3026,"apply.run_wall_time_us":48262,"num_ops":1,"prepare.queue_time_us":1367,"prepare.run_cpu_time_us":4301,"prepare.run_wall_time_us":4310,"replication_time_us":89034,"thread_start_us":68963,"threads_started":1}]]}
I20260809 15:51:58.722376 12320 consensus_queue.cc:1035] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.005s
I20260809 15:51:58.740658 12320 consensus_queue.cc:1035] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:58.773073 10737 catalog_manager.cc:5495] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:58.775580 10737 catalog_manager.cc:5495] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 1 to 2, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 2 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:58.823655 10931 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:51:58.824261 10931 raft_consensus.cc:595] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:51:58.842018 12447 raft_consensus.cc:979] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:51:58.842535 12447 raft_consensus.cc:1067] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:51:58.846027 12319 raft_consensus.cc:979] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:51:58.846664 12359 raft_consensus.cc:1067] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:51:58.851629 11012 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50542
I20260809 15:51:58.852306 11012 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:58.852705 11012 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:51:58.863657 11016 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50542
I20260809 15:51:58.864243 11016 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:58.864593 11016 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:58.869359 10929 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:51:58.871066 10929 raft_consensus.cc:595] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Received request to transfer leadership to TS c7aa762318cb4e97875333f10679a9f9
I20260809 15:51:58.872701 11012 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:58.884989 11012 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:58.891227 11016 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:58.892531 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260809 15:51:58.894788 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:51:58.896391 11016 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:51:58.899657 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:51:58.906445 10737 catalog_manager.cc:5495] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 1 to 2, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:58.912691 12435 raft_consensus.cc:979] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:51:58.913313 12441 raft_consensus.cc:1067] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:51:58.927632 10931 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54224
I20260809 15:51:58.928195 10931 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:51:58.928521 10931 raft_consensus.cc:3037] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:58.936040 10931 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:58.939838 10929 raft_consensus.cc:1261] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:51:58.947618 12441 consensus_queue.cc:1035] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:51:58.964987 10923 consensus_queue.cc:237] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:58.970821 10733 catalog_manager.cc:5495] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 2 to 3, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 3 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:58.971797 11014 raft_consensus.cc:1261] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:51:58.975838 11012 raft_consensus.cc:1261] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:58.977258 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:58.979126 10839 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:51:58.980022 12179 consensus_queue.cc:1035] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:51:58.990974 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:58.991544 10847 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:58.993634 10839 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:51:58.995669 12359 raft_consensus.cc:2932] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:59.003309 11012 raft_consensus.cc:2932] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:59.004941 11637 raft_consensus.cc:1261] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:51:59.010150 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:59.010804 10923 raft_consensus.cc:3032] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:51:59.011201 10923 raft_consensus.cc:729] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:59.012109 10923 consensus_queue.cc:260] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:59.014636 12179 consensus_queue.cc:1035] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:51:59.011278 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:59.020159 10927 raft_consensus.cc:3032] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:59.020516 10927 raft_consensus.cc:729] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:59.021260 10927 consensus_queue.cc:260] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:59.021214 11637 raft_consensus.cc:2932] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:59.022857 10927 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:59.026182 10923 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:51:59.026469 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 0f891fe62df749108fa553cb0661a874 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:59.031184 10844 consensus_queue.cc:237] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:59.032953 10931 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:59.042533 10733 catalog_manager.cc:5495] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:59.044298 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:59.044829 11016 raft_consensus.cc:3037] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:59.049497 11016 raft_consensus.cc:2454] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
W20260809 15:51:59.051082 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 49503.9us
I20260809 15:51:59.051469 10891 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:59.052901 10923 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:51:59.053362 10828 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f891fe62df749108fa553cb0661a874 with delete_type TABLET_DATA_TOMBSTONED (TS a53441872c0c416b8e2f26f52399f536 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45572
I20260809 15:51:59.056834 12447 raft_consensus.cc:2781] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:59.057350 12447 raft_consensus.cc:686] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:59.058189 12447 consensus_queue.cc:237] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:59.058599 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:51:59.059146 10842 raft_consensus.cc:3032] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:59.059461 10842 raft_consensus.cc:729] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:51:59.060153 10842 consensus_queue.cc:260] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:59.061481 10842 raft_consensus.cc:3037] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:59.053336 10927 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:51:59.077880 10842 raft_consensus.cc:2454] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:51:59.002293 10847 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:51:59.082404 12457 tablet_replica.cc:330] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536: stopping tablet replica
I20260809 15:51:59.093086 10978 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:59.094107 12170 raft_consensus.cc:2781] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:51:59.094604 12170 raft_consensus.cc:686] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:59.096145 12170 consensus_queue.cc:237] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:59.109630 10978 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:51:59.110934 12457 raft_consensus.cc:2229] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:59.110348 10733 catalog_manager.cc:5495] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 1 to 2, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:51:59.116425 12457 raft_consensus.cc:2258] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:59.124658 12435 consensus_queue.cc:1035] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:59.131412 12322 raft_consensus.cc:2781] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:59.131920 12322 raft_consensus.cc:686] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:51:59.150619 10931 raft_consensus.cc:1261] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:59.219691 12406 consensus_queue.cc:1035] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.023s
I20260809 15:51:59.228273 11636 raft_consensus.cc:1261] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:59.208753 12322 consensus_queue.cc:237] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:51:59.256537 12411 raft_consensus.cc:979] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3: : Instructing follower c7aa762318cb4e97875333f10679a9f9 to start an election
W20260809 15:51:59.270870 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 112937us
I20260809 15:51:59.262678 12359 raft_consensus.cc:1067] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Signalling peer c7aa762318cb4e97875333f10679a9f9 to start an election
W20260809 15:51:59.322335 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 101495us
W20260809 15:51:59.332028 12459 log.cc:924] Time spent T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.066s user 0.001s sys 0.000s
I20260809 15:51:59.331867 11636 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c"
dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:41488
I20260809 15:51:59.334283 12457 ts_tablet_manager.cc:1905] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:59.334415 11636 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
W20260809 15:51:59.334609 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 103609us
I20260809 15:51:59.335000 11636 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:59.344475 10733 catalog_manager.cc:5495] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 2 to 3, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 3 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:59.348977 11636 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:59.371852 11636 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:51:59.372681 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:51:59.373422 10931 raft_consensus.cc:3032] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:51:59.373777 10931 raft_consensus.cc:729] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:51:59.374653 10931 consensus_queue.cc:260] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:59.378935 10931 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:59.385293 10737 catalog_manager.cc:5495] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 1 to 2, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 2 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:51:59.407629 10931 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:51:59.423869 11606 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:51:59.424803 12448 raft_consensus.cc:2781] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:51:59.427024 12448 raft_consensus.cc:686] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:51:59.428254 12448 consensus_queue.cc:237] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:51:59.431546 12462 consensus_queue.cc:1035] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.002s
I20260809 15:51:59.444926 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:51:59.445480 11012 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:51:59.453388 11012 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:51:59.480225 10842 raft_consensus.cc:1261] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:59.481729 12359 consensus_queue.cc:1035] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:59.502857 12457 ts_tablet_manager.cc:1918] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:51:59.504263 12457 log.cc:1196] T 0f891fe62df749108fa553cb0661a874 P a53441872c0c416b8e2f26f52399f536: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-0-root/wals/0f891fe62df749108fa553cb0661a874
I20260809 15:51:59.519492 10722 catalog_manager.cc:4841] TS a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): tablet 0f891fe62df749108fa553cb0661a874 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:59.546949 12440 raft_consensus.cc:2932] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:59.563362 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 8feeb57bf5cf4fd1bb17ab7f575222d0 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:51:59.572697 11636 raft_consensus.cc:2932] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:59.586937 10927 raft_consensus.cc:2932] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:51:59.619661 11012 raft_consensus.cc:1261] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:59.624922 12447 consensus_queue.cc:1035] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:51:59.657464 10847 raft_consensus.cc:1261] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:51:59.659228 12322 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:59.667618 10931 raft_consensus.cc:1261] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:51:59.668244 10733 catalog_manager.cc:5495] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 1 to 2, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 2 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:51:59.668885 12322 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:51:59.697569 10736 catalog_manager.cc:5495] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index 4 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260809 15:51:59.709442 12476 log.cc:924] Time spent T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.063s user 0.001s sys 0.000s
I20260809 15:51:59.720005 10998 tablet_service.cc:1514] Processing DeleteTablet for tablet 8feeb57bf5cf4fd1bb17ab7f575222d0 with delete_type TABLET_DATA_TOMBSTONED (TS d74944fa9fa14ec78b0640ea709ff3aa not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:50526
I20260809 15:51:59.737568 12481 tablet_replica.cc:330] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa: stopping tablet replica
I20260809 15:51:59.738759 12481 raft_consensus.cc:2229] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 15:51:59.750339 12481 raft_consensus.cc:2258] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:51:59.753311 12481 ts_tablet_manager.cc:1905] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:51:59.776180 12481 ts_tablet_manager.cc:1918] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260809 15:51:59.776682 12481 log.cc:1196] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P d74944fa9fa14ec78b0640ea709ff3aa: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-2-root/wals/8feeb57bf5cf4fd1bb17ab7f575222d0
I20260809 15:51:59.778833 10719 catalog_manager.cc:4841] TS d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): tablet 8feeb57bf5cf4fd1bb17ab7f575222d0 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:51:59.786037 10924 raft_consensus.cc:1261] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:59.790766 12469 consensus_queue.cc:1035] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:59.800302 10842 raft_consensus.cc:1261] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:51:59.804555 12469 consensus_queue.cc:1035] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:51:59.897974 10927 raft_consensus.cc:1261] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:59.899266 12448 consensus_queue.cc:1035] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:51:59.927877 11012 raft_consensus.cc:1261] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:51:59.929490 12463 consensus_queue.cc:1035] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:00.123241 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:00.162667 11012 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:52:00.163290 11012 raft_consensus.cc:595] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:52:00.165735 10840 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:52:00.167500 10840 raft_consensus.cc:595] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:00.170055 11012 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:52:00.170554 11012 raft_consensus.cc:595] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:00.172518 11012 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:52:00.172911 11012 raft_consensus.cc:595] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:52:00.174194 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260809 15:52:00.175319 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:52:00.175874 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:00.235651 12469 raft_consensus.cc:979] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:00.236200 12467 raft_consensus.cc:1067] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:00.239128 10840 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45578
I20260809 15:52:00.239684 10840 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:00.240042 10840 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:00.245501 10840 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:00.247948 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:00.247565 10840 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:00.248540 10924 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:00.248637 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:00.249382 11018 raft_consensus.cc:3032] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:00.249601 11018 raft_consensus.cc:729] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:00.250200 11018 consensus_queue.cc:260] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:00.251026 11018 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:00.257373 10924 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:00.258589 10811 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:00.259225 11018 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:00.259513 12435 raft_consensus.cc:2781] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:00.260939 12435 raft_consensus.cc:686] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:00.261968 12435 consensus_queue.cc:237] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:00.272538 10733 catalog_manager.cc:5495] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 2 to 3, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 3 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:00.484262 12467 raft_consensus.cc:979] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:00.484804 12467 raft_consensus.cc:1067] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:00.487000 10840 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45578
I20260809 15:52:00.488260 10840 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:00.492551 10840 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:00.500998 10840 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:00.505224 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:00.505550 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:00.505784 11018 raft_consensus.cc:3032] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:00.506139 11018 raft_consensus.cc:729] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:00.506273 11639 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:00.507426 11018 consensus_queue.cc:260] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:00.508472 11018 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:00.513013 11639 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:00.514182 10811 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:00.514775 11018 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:00.504948 10840 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:00.515532 12433 raft_consensus.cc:2781] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:52:00.517342 12433 raft_consensus.cc:686] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:00.518458 12433 consensus_queue.cc:237] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:00.530498 10733 catalog_manager.cc:5495] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 1 to 2, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:00.544057 12469 raft_consensus.cc:979] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:00.544486 12467 raft_consensus.cc:1067] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:00.545843 10924 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54236
I20260809 15:52:00.546329 10924 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:00.546602 10924 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:00.551201 10924 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:00.552923 10924 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:00.553840 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:00.554507 11012 raft_consensus.cc:3032] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:00.554774 11012 raft_consensus.cc:729] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:00.555459 11012 consensus_queue.cc:260] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:00.556545 12435 raft_consensus.cc:979] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:00.556653 11012 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:00.557099 12435 raft_consensus.cc:1067] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:00.558928 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:00.559521 11639 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:00.565862 11639 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:00.565991 10924 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54224
I20260809 15:52:00.566535 10924 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:00.566968 10924 raft_consensus.cc:3037] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:00.567050 10893 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:00.568001 12359 raft_consensus.cc:2781] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:52:00.568516 12359 raft_consensus.cc:686] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:00.569386 12359 consensus_queue.cc:237] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:00.572752 10924 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:00.574358 10924 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:00.575305 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:00.575414 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:00.576063 10840 raft_consensus.cc:3032] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:00.576174 11014 raft_consensus.cc:3037] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:00.576467 10840 raft_consensus.cc:729] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:00.577212 10840 consensus_queue.cc:260] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:00.577970 10840 raft_consensus.cc:3037] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:00.585505 10840 raft_consensus.cc:2454] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:00.588073 10893 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:00.588855 12447 raft_consensus.cc:2781] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:52:00.589304 12447 raft_consensus.cc:686] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:00.590117 12447 consensus_queue.cc:237] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:00.585945 11014 raft_consensus.cc:2454] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:00.595132 11012 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:00.600263 10733 catalog_manager.cc:5495] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 1 to 2, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:00.625115 10733 catalog_manager.cc:5495] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 1 to 2, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:00.740829 11012 raft_consensus.cc:1261] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:52:00.742527 12492 consensus_queue.cc:1035] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:00.746658 10924 raft_consensus.cc:1261] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:52:00.748109 12496 consensus_queue.cc:1035] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:00.837369 11012 consensus_queue.cc:237] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:00.845095 10924 raft_consensus.cc:1261] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 15:52:00.845752 10978 consensus_peers.cc:487] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: b6d399a7394c4a59b90a308aedfdf899. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:52:00.846670 10842 raft_consensus.cc:1261] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:52:00.847930 12469 consensus_queue.cc:1035] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:00.849504 12469 consensus_queue.cc:1035] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:00.904446 12469 raft_consensus.cc:2932] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:00.912144 10924 raft_consensus.cc:2932] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:00.912415 10842 raft_consensus.cc:2932] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:00.922676 10840 consensus_queue.cc:237] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:00.938735 11012 raft_consensus.cc:1261] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:52:00.941493 12495 consensus_queue.cc:1035] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:00.937773 10734 catalog_manager.cc:5495] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260809 15:52:00.946797 10811 consensus_peers.cc:487] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 6d4d6dee78914014adab40a391b02955. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:52:00.952672 11012 raft_consensus.cc:1261] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:00.953235 10931 raft_consensus.cc:1261] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:52:00.959441 12496 consensus_queue.cc:1035] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:00.960887 12495 consensus_queue.cc:1035] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:00.969978 12496 raft_consensus.cc:2932] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:00.985234 11018 raft_consensus.cc:2932] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:00.997534 10840 consensus_queue.cc:237] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:01.008037 11639 raft_consensus.cc:1261] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:01.010296 12496 consensus_queue.cc:1035] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:01.027661 10734 catalog_manager.cc:5495] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:01.045465 11012 raft_consensus.cc:1261] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:52:01.049049 12447 consensus_queue.cc:1035] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:01.055842 10931 raft_consensus.cc:2932] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
W20260809 15:52:01.061592 10811 consensus_peers.cc:487] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e6e09d263f96440a99c99d5a5c6ea477. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:52:01.071754 11014 raft_consensus.cc:1261] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260809 15:52:01.074069 12492 consensus_queue.cc:1035] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:52:01.086274 10842 raft_consensus.cc:1261] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:52:01.092949 12369 consensus_queue.cc:1035] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:52:01.114609 11639 raft_consensus.cc:1261] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:01.115777 12447 consensus_queue.cc:1035] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:01.127123 10924 raft_consensus.cc:1261] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260809 15:52:01.133261 12496 consensus_queue.cc:1035] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:52:01.168097 12496 raft_consensus.cc:2932] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:01.177316 11018 raft_consensus.cc:1261] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:01.182920 12369 consensus_queue.cc:1035] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:01.190361 11018 raft_consensus.cc:2932] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:01.184432 10931 raft_consensus.cc:2932] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:01.208887 10840 consensus_queue.cc:237] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
W20260809 15:52:01.243168 10811 consensus_peers.cc:487] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 369529a0023c4cf4a709bb1a480052b4. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:52:01.251123 10736 catalog_manager.cc:5495] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:01.257998 11018 raft_consensus.cc:1261] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:52:01.261816 12496 consensus_queue.cc:1035] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:01.262311 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:01.265125 10931 raft_consensus.cc:1261] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:52:01.307286 12492 consensus_queue.cc:1035] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:01.319666 12492 raft_consensus.cc:2932] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:01.330893 10927 raft_consensus.cc:2932] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:01.334815 12525 ts_tablet_manager.cc:927] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615)
I20260809 15:52:01.365592 10733 catalog_manager.cc:5495] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:01.371769 12525 tablet_copy_client.cc:316] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.1:46615
I20260809 15:52:01.395386 10858 tablet_copy_service.cc:140] P a53441872c0c416b8e2f26f52399f536: Received BeginTabletCopySession request for tablet 6d4d6dee78914014adab40a391b02955 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:54342)
I20260809 15:52:01.395956 10858 tablet_copy_service.cc:161] P a53441872c0c416b8e2f26f52399f536: Beginning new tablet copy session on tablet 6d4d6dee78914014adab40a391b02955 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:54342: session id = c7aa762318cb4e97875333f10679a9f9-6d4d6dee78914014adab40a391b02955
I20260809 15:52:01.399374 10842 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:52:01.399971 10842 raft_consensus.cc:595] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:01.403461 10927 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:01.403988 10927 raft_consensus.cc:595] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:01.407028 10842 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:52:01.408752 10842 raft_consensus.cc:595] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:01.414837 10924 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:01.415372 10924 raft_consensus.cc:595] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:01.417171 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260809 15:52:01.419008 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:52:01.419597 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:01.424079 10858 tablet_copy_source_session.cc:205] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:52:01.427927 11018 raft_consensus.cc:2932] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:01.429381 12525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6d4d6dee78914014adab40a391b02955. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:52:01.454522 12528 ts_tablet_manager.cc:927] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:01.465848 12369 raft_consensus.cc:979] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:01.466842 12520 raft_consensus.cc:1067] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:01.469162 11018 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50542
I20260809 15:52:01.469679 11018 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:01.472172 11018 raft_consensus.cc:3037] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:01.484666 12528 tablet_copy_client.cc:316] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.3:39223
I20260809 15:52:01.486810 11027 tablet_copy_service.cc:140] P d74944fa9fa14ec78b0640ea709ff3aa: Received BeginTabletCopySession request for tablet b6d399a7394c4a59b90a308aedfdf899 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:39978)
I20260809 15:52:01.487365 11027 tablet_copy_service.cc:161] P d74944fa9fa14ec78b0640ea709ff3aa: Beginning new tablet copy session on tablet b6d399a7394c4a59b90a308aedfdf899 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:39978: session id = c7aa762318cb4e97875333f10679a9f9-b6d399a7394c4a59b90a308aedfdf899
I20260809 15:52:01.494339 12531 ts_tablet_manager.cc:927] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615)
I20260809 15:52:01.505066 11027 tablet_copy_source_session.cc:205] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:52:01.512017 12528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b6d399a7394c4a59b90a308aedfdf899. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:52:01.529374 12525 tablet_copy_client.cc:791] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:52:01.530027 12525 tablet_copy_client.cc:655] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:52:01.482681 11018 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:01.534394 12525 tablet_copy_client.cc:531] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:52:01.541277 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:01.541873 10847 raft_consensus.cc:3037] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:01.542416 11018 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:01.543648 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:01.544203 10924 raft_consensus.cc:3032] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:01.544463 10924 raft_consensus.cc:729] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:01.545136 10924 consensus_queue.cc:260] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:01.549305 10924 raft_consensus.cc:3037] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:01.553447 12525 tablet_bootstrap.cc:492] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:52:01.586943 12528 tablet_copy_client.cc:791] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:52:01.587761 12528 tablet_copy_client.cc:655] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:52:01.593542 12528 tablet_copy_client.cc:531] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:52:01.597882 10847 raft_consensus.cc:2454] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:01.586804 10924 raft_consensus.cc:2454] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:01.602118 10978 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:01.603474 12506 raft_consensus.cc:2781] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:01.603968 12506 raft_consensus.cc:686] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:01.604765 12506 consensus_queue.cc:237] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:01.610749 12531 tablet_copy_client.cc:316] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.1:46615
I20260809 15:52:01.622331 10858 tablet_copy_service.cc:140] P a53441872c0c416b8e2f26f52399f536: Received BeginTabletCopySession request for tablet e6e09d263f96440a99c99d5a5c6ea477 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:54342)
I20260809 15:52:01.622882 10858 tablet_copy_service.cc:161] P a53441872c0c416b8e2f26f52399f536: Beginning new tablet copy session on tablet e6e09d263f96440a99c99d5a5c6ea477 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:54342: session id = c7aa762318cb4e97875333f10679a9f9-e6e09d263f96440a99c99d5a5c6ea477
I20260809 15:52:01.625916 12528 tablet_bootstrap.cc:492] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:52:01.627498 10734 catalog_manager.cc:5495] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 2 to 3, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 3 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:52:01.640498 10858 tablet_copy_source_session.cc:205] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:52:01.646109 12531 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6e09d263f96440a99c99d5a5c6ea477. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:52:01.648397 12525 tablet_bootstrap.cc:492] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:52:01.649257 12525 tablet_bootstrap.cc:492] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:52:01.680956 12525 ts_tablet_manager.cc:1397] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.097s user 0.020s sys 0.010s
I20260809 15:52:01.684671 12525 raft_consensus.cc:348] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:01.685657 12525 raft_consensus.cc:729] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:52:01.686332 12525 consensus_queue.cc:260] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:01.689543 12528 tablet_bootstrap.cc:492] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:52:01.692651 12528 tablet_bootstrap.cc:492] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:52:01.693059 12525 ts_tablet_manager.cc:1428] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260809 15:52:01.693393 12528 ts_tablet_manager.cc:1397] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.068s user 0.022s sys 0.005s
I20260809 15:52:01.694667 10858 tablet_copy_service.cc:342] P a53441872c0c416b8e2f26f52399f536: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-6d4d6dee78914014adab40a391b02955 received from {username='slave'} at 127.0.0.1:54342
I20260809 15:52:01.695081 10858 tablet_copy_service.cc:433] P a53441872c0c416b8e2f26f52399f536: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-6d4d6dee78914014adab40a391b02955 on tablet 6d4d6dee78914014adab40a391b02955 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:01.697997 12528 raft_consensus.cc:348] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:01.698869 12528 raft_consensus.cc:729] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:52:01.699514 12528 consensus_queue.cc:260] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:01.700662 12531 tablet_copy_client.cc:791] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:52:01.701208 12531 tablet_copy_client.cc:655] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:52:01.702303 12528 ts_tablet_manager.cc:1428] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.009s user 0.005s sys 0.003s
I20260809 15:52:01.703809 11027 tablet_copy_service.cc:342] P d74944fa9fa14ec78b0640ea709ff3aa: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-b6d399a7394c4a59b90a308aedfdf899 received from {username='slave'} at 127.0.0.1:39978
I20260809 15:52:01.704181 11027 tablet_copy_service.cc:433] P d74944fa9fa14ec78b0640ea709ff3aa: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-b6d399a7394c4a59b90a308aedfdf899 on tablet b6d399a7394c4a59b90a308aedfdf899 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:01.704751 12531 tablet_copy_client.cc:531] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:52:01.713186 12531 tablet_bootstrap.cc:492] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:52:01.741595 12528 ts_tablet_manager.cc:927] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615)
I20260809 15:52:01.742619 12433 raft_consensus.cc:979] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:01.743152 12433 raft_consensus.cc:1067] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:01.761752 10924 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54224
I20260809 15:52:01.762398 10924 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:01.762761 10924 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:01.804211 12369 raft_consensus.cc:979] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:01.806211 12369 raft_consensus.cc:1067] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:01.807608 11018 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50542
I20260809 15:52:01.808269 11018 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:01.808691 11018 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:01.816319 10924 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:01.822173 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:01.822795 10840 raft_consensus.cc:3032] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:01.823325 10840 raft_consensus.cc:729] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
W20260809 15:52:01.822890 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 45103.1us
I20260809 15:52:01.826367 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:01.826972 11636 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:01.824144 10840 consensus_queue.cc:260] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:01.858991 11636 raft_consensus.cc:2454] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:01.860303 10893 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:01.861083 12369 raft_consensus.cc:2781] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:01.861645 12369 raft_consensus.cc:686] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:01.862551 12369 consensus_queue.cc:237] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:01.863546 11639 raft_consensus.cc:1203] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:52:01.861127 10840 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:01.876818 10924 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:01.890702 10734 catalog_manager.cc:5495] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 2 to 3, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 3 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:01.893896 12433 raft_consensus.cc:979] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:01.894484 12495 raft_consensus.cc:1067] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:01.902748 10840 raft_consensus.cc:2454] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:01.904328 11010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50528
I20260809 15:52:01.904965 11010 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:01.905459 11010 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:01.908434 11018 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:01.913218 12528 tablet_copy_client.cc:316] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.1:46615
I20260809 15:52:01.914654 10858 tablet_copy_service.cc:140] P a53441872c0c416b8e2f26f52399f536: Received BeginTabletCopySession request for tablet 369529a0023c4cf4a709bb1a480052b4 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:54342)
I20260809 15:52:01.915185 10858 tablet_copy_service.cc:161] P a53441872c0c416b8e2f26f52399f536: Beginning new tablet copy session on tablet 369529a0023c4cf4a709bb1a480052b4 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:54342: session id = c7aa762318cb4e97875333f10679a9f9-369529a0023c4cf4a709bb1a480052b4
I20260809 15:52:01.921010 12531 tablet_bootstrap.cc:492] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:52:01.921182 11018 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:01.921960 12531 tablet_bootstrap.cc:492] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:52:01.922755 12531 ts_tablet_manager.cc:1397] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.210s user 0.055s sys 0.047s
I20260809 15:52:01.925510 12531 raft_consensus.cc:348] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:01.926380 12531 raft_consensus.cc:729] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:52:01.927021 12531 consensus_queue.cc:260] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:01.930063 12531 ts_tablet_manager.cc:1428] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260809 15:52:01.933128 10857 tablet_copy_service.cc:342] P a53441872c0c416b8e2f26f52399f536: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-e6e09d263f96440a99c99d5a5c6ea477 received from {username='slave'} at 127.0.0.1:54342
I20260809 15:52:01.933514 10857 tablet_copy_service.cc:433] P a53441872c0c416b8e2f26f52399f536: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-e6e09d263f96440a99c99d5a5c6ea477 on tablet e6e09d263f96440a99c99d5a5c6ea477 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:01.944861 11010 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:01.946866 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:01.959597 10931 raft_consensus.cc:3032] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:01.959914 10931 raft_consensus.cc:729] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:01.960575 10931 consensus_queue.cc:260] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:01.961759 10931 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:01.962481 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:01.962998 11639 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:01.970218 10931 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:01.952495 10858 tablet_copy_source_session.cc:205] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:52:01.971323 10978 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:01.972088 12507 raft_consensus.cc:2781] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:01.972527 12507 raft_consensus.cc:686] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:01.973349 12507 consensus_queue.cc:237] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:01.975942 12528 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 369529a0023c4cf4a709bb1a480052b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:52:01.973593 11639 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:01.985121 11639 raft_consensus.cc:1203] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:52:01.986176 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:01.987304 10840 raft_consensus.cc:3032] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:01.987654 10840 raft_consensus.cc:729] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:01.988454 10840 consensus_queue.cc:260] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:01.989588 10840 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:01.991940 12528 tablet_copy_client.cc:791] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:52:01.992518 12528 tablet_copy_client.cc:655] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:52:01.983251 11010 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:01.999011 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:01.999606 11636 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:02.000625 10734 catalog_manager.cc:5495] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 2 to 3, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 3 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:02.001433 10840 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:02.005852 12528 tablet_copy_client.cc:531] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:52:02.009138 10978 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:02.009829 11636 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:02.010980 12538 raft_consensus.cc:2781] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:52:02.011451 12538 raft_consensus.cc:686] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:02.012316 12538 consensus_queue.cc:237] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:02.016439 12528 tablet_bootstrap.cc:492] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:52:02.037539 10734 catalog_manager.cc:5495] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 1 to 2, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 2 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:02.056994 12528 tablet_bootstrap.cc:492] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:52:02.058391 12528 tablet_bootstrap.cc:492] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:52:02.059125 12528 ts_tablet_manager.cc:1397] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.043s user 0.019s sys 0.014s
I20260809 15:52:02.061929 12528 raft_consensus.cc:348] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:02.062773 12528 raft_consensus.cc:729] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:52:02.063397 12528 consensus_queue.cc:260] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:02.069178 12528 ts_tablet_manager.cc:1428] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.010s user 0.001s sys 0.005s
I20260809 15:52:02.070657 10858 tablet_copy_service.cc:342] P a53441872c0c416b8e2f26f52399f536: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-369529a0023c4cf4a709bb1a480052b4 received from {username='slave'} at 127.0.0.1:54342
I20260809 15:52:02.071096 10858 tablet_copy_service.cc:433] P a53441872c0c416b8e2f26f52399f536: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-369529a0023c4cf4a709bb1a480052b4 on tablet 369529a0023c4cf4a709bb1a480052b4 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:02.123327 10931 raft_consensus.cc:1261] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:52:02.124845 12538 consensus_queue.cc:1035] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:02.143635 10840 raft_consensus.cc:1261] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:52:02.146288 12503 consensus_queue.cc:1035] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:02.239490 12495 raft_consensus.cc:1050] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:52:02.241611 12495 consensus_queue.cc:237] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:02.255875 11636 raft_consensus.cc:1261] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:02.257934 10924 raft_consensus.cc:1261] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:02.258399 11010 raft_consensus.cc:1261] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:02.257555 12495 consensus_queue.cc:1035] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260809 15:52:02.261516 12433 consensus_queue.cc:1035] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:02.262679 12492 consensus_queue.cc:1035] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:02.267619 11636 raft_consensus.cc:1203] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:52:02.303817 12492 raft_consensus.cc:2932] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:02.308708 11636 raft_consensus.cc:2932] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:02.319639 11639 raft_consensus.cc:1203] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20260809 15:52:02.319823 11010 raft_consensus.cc:2932] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:02.333134 10924 raft_consensus.cc:2932] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:02.347549 10839 raft_consensus.cc:1261] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:02.352221 12359 consensus_queue.cc:1035] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:02.371522 10736 catalog_manager.cc:5495] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:02.407218 11639 raft_consensus.cc:1261] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:02.413533 12369 consensus_queue.cc:1035] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 15:52:02.432889 10842 consensus_queue.cc:237] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:02.437177 10924 raft_consensus.cc:1261] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:02.450510 11636 raft_consensus.cc:1261] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:52:02.465984 11637 raft_consensus.cc:1261] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:02.476151 12492 consensus_queue.cc:1035] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:52:02.482196 12506 consensus_queue.cc:1035] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260809 15:52:02.491859 12433 raft_consensus.cc:2932] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:02.494829 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:02.496120 11637 raft_consensus.cc:2932] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:02.506763 11010 raft_consensus.cc:1261] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:52:02.507936 12506 consensus_queue.cc:1035] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:02.508205 12548 consensus_queue.cc:1035] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:02.512352 12503 raft_consensus.cc:1050] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:52:02.512984 11010 raft_consensus.cc:2932] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:02.515398 12503 consensus_queue.cc:237] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:02.537570 10733 catalog_manager.cc:5495] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:02.552063 10913 tablet_service.cc:1514] Processing DeleteTablet for tablet 6d4d6dee78914014adab40a391b02955 with delete_type TABLET_DATA_TOMBSTONED (TS 20730d75ad954a82b126148f8152a9f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54216
I20260809 15:52:02.557006 11637 raft_consensus.cc:1261] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:02.560252 12538 consensus_queue.cc:1035] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:02.576303 12562 tablet_replica.cc:330] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3: stopping tablet replica
I20260809 15:52:02.577175 10842 raft_consensus.cc:1261] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:02.587242 12507 consensus_queue.cc:1035] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:02.591905 10842 raft_consensus.cc:1261] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:02.593323 12503 consensus_queue.cc:1035] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:52:02.592342 11639 raft_consensus.cc:1261] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:02.597548 10924 raft_consensus.cc:1261] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:02.598693 12564 consensus_queue.cc:1035] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:52:02.600574 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 6d4d6dee78914014adab40a391b02955 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:52:02.602149 12507 consensus_queue.cc:1035] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260809 15:52:02.657851 11010 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:52:02.662043 11010 raft_consensus.cc:595] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:02.666685 11010 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "908409e5747b470cabbc8aed443397f7"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:52:02.667217 11010 raft_consensus.cc:595] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:52:02.678290 11010 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "93303d409ab940409d87013868417907"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:52:02.678829 11010 raft_consensus.cc:595] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:02.683660 10927 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:02.683883 12562 raft_consensus.cc:2229] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:52:02.684203 10927 raft_consensus.cc:595] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:52:02.684685 12562 raft_consensus.cc:2258] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:52:02.687348 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260809 15:52:02.689159 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:52:02.689774 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:02.694931 12562 ts_tablet_manager.cc:1905] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:52:02.757740 12562 ts_tablet_manager.cc:1918] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:52:02.758349 12562 log.cc:1196] T 6d4d6dee78914014adab40a391b02955 P 20730d75ad954a82b126148f8152a9f3: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-1-root/wals/6d4d6dee78914014adab40a391b02955
I20260809 15:52:02.898718 12548 raft_consensus.cc:1050] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:52:03.231791 12576 raft_consensus.cc:979] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:03.232338 12576 raft_consensus.cc:1067] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:03.254341 10722 catalog_manager.cc:4841] TS 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): tablet 6d4d6dee78914014adab40a391b02955 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
W20260809 15:52:03.263513 10924 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.651s user 0.000s sys 0.014s
W20260809 15:52:03.263876 10924 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.652s user 0.000s sys 0.014s
I20260809 15:52:03.311887 12548 consensus_queue.cc:237] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:03.349138 12582 raft_consensus.cc:979] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:03.349645 12582 raft_consensus.cc:1067] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:03.352357 12586 raft_consensus.cc:979] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:03.354112 12586 raft_consensus.cc:1067] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
W20260809 15:52:03.355064 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 71962.9us
I20260809 15:52:03.363538 12582 raft_consensus.cc:2932] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
W20260809 15:52:03.367934 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 76237.1us
I20260809 15:52:03.376518 12507 raft_consensus.cc:979] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:03.377029 12586 raft_consensus.cc:1067] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:03.377768 10847 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45594
I20260809 15:52:03.378368 10847 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:03.378742 10847 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:03.390151 10839 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "908409e5747b470cabbc8aed443397f7"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45578
I20260809 15:52:03.390774 10839 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:03.385637 10847 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:03.391120 10839 raft_consensus.cc:3037] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:03.397789 10840 raft_consensus.cc:2932] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:03.402132 10847 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:03.406380 10924 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "93303d409ab940409d87013868417907"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54236
I20260809 15:52:03.406880 10924 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:03.397773 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:03.407300 10924 raft_consensus.cc:3037] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:03.407582 10929 raft_consensus.cc:3032] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:03.407970 10929 raft_consensus.cc:729] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:03.408712 10929 consensus_queue.cc:260] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:03.409853 10929 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:03.409833 10839 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:03.414110 10924 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:03.415827 10924 leader_election.cc:288] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:03.415457 10733 catalog_manager.cc:5495] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:03.416661 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:03.417196 11010 raft_consensus.cc:3032] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:03.417454 11010 raft_consensus.cc:729] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:03.418182 11010 consensus_queue.cc:260] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:03.417912 10931 raft_consensus.cc:2932] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:03.419068 11010 raft_consensus.cc:3037] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:03.421075 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:03.421808 10847 raft_consensus.cc:3037] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:03.426069 11010 raft_consensus.cc:2454] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:03.427227 10891 leader_election.cc:302] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:03.428130 10929 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:03.428207 12557 raft_consensus.cc:2781] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:52:03.428874 12557 raft_consensus.cc:686] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:03.430176 10923 raft_consensus.cc:1261] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:03.429703 12557 consensus_queue.cc:237] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:03.431386 12580 consensus_queue.cc:1035] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:03.434283 10924 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54236
I20260809 15:52:03.434831 10924 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:03.435168 10924 raft_consensus.cc:3037] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:03.435887 10811 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:03.436743 12580 raft_consensus.cc:2781] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:03.437178 12580 raft_consensus.cc:686] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:03.439388 10847 raft_consensus.cc:2454] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:03.440083 10842 raft_consensus.cc:1226] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:03.437978 12580 consensus_queue.cc:237] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:03.443549 10839 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:03.444399 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:03.444921 10929 raft_consensus.cc:3037] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:03.449265 10924 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:03.451076 10924 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:03.451834 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:03.452361 11010 raft_consensus.cc:3032] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:03.452649 11010 raft_consensus.cc:729] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:03.453231 11010 consensus_queue.cc:260] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:03.457299 11010 raft_consensus.cc:3037] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:03.460690 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:03.461457 10840 raft_consensus.cc:3037] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:03.462823 11010 raft_consensus.cc:2454] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:03.464161 10891 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:03.466318 10929 raft_consensus.cc:2454] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:03.466563 12557 raft_consensus.cc:2781] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:03.467216 10811 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:03.468112 12557 raft_consensus.cc:686] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:03.468194 12548 raft_consensus.cc:2781] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:52:03.469602 12548 raft_consensus.cc:686] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:03.470889 12557 consensus_queue.cc:237] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:03.471150 12548 consensus_queue.cc:237] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:03.487308 11010 raft_consensus.cc:1261] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:03.469764 10840 raft_consensus.cc:2454] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:03.497730 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:03.498404 11018 raft_consensus.cc:3032] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:03.498704 11018 raft_consensus.cc:729] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:03.499467 11018 consensus_queue.cc:260] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:03.500721 11018 raft_consensus.cc:3037] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:03.495720 12492 consensus_queue.cc:1035] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.008s
I20260809 15:52:03.533703 11018 raft_consensus.cc:2454] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
W20260809 15:52:03.567728 12598 log.cc:924] Time spent T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.072s user 0.001s sys 0.000s
I20260809 15:52:03.584848 10737 catalog_manager.cc:5495] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 2 to 3, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 3 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:03.587349 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:03.587991 11639 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:03.601679 12580 raft_consensus.cc:1050] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:52:03.601975 11637 raft_consensus.cc:1261] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:03.603022 11639 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:03.606977 12580 consensus_queue.cc:237] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:03.610333 12433 raft_consensus.cc:2932] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:03.612511 11014 raft_consensus.cc:2932] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:03.629130 10929 raft_consensus.cc:1261] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:52:03.631309 12580 consensus_queue.cc:1035] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:52:03.635098 11018 raft_consensus.cc:1261] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:52:03.633821 12580 consensus_queue.cc:1035] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.002s
I20260809 15:52:03.636528 12495 consensus_queue.cc:1035] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:52:03.641196 11639 raft_consensus.cc:1261] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:52:03.644371 11636 raft_consensus.cc:2932] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:03.644187 10736 catalog_manager.cc:5495] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 1 to 2, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:03.657179 11014 consensus_queue.cc:237] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:03.666303 12580 consensus_queue.cc:1035] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.002s
I20260809 15:52:03.673770 10923 raft_consensus.cc:1261] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:52:03.672320 10733 catalog_manager.cc:5495] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:03.684283 10923 raft_consensus.cc:2932] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:03.690518 12503 consensus_queue.cc:1035] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:52:03.702730 11636 raft_consensus.cc:1261] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:52:03.710644 10737 catalog_manager.cc:5495] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 1 to 2, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:03.714011 10924 raft_consensus.cc:2932] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:03.707485 11637 raft_consensus.cc:2932] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:03.716295 12564 consensus_queue.cc:1035] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.010s
I20260809 15:52:03.711294 12582 raft_consensus.cc:2932] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:03.745307 11642 raft_consensus.cc:2932] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:03.756402 10719 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet b6d399a7394c4a59b90a308aedfdf899 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:52:03.762975 12495 raft_consensus.cc:2932] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:03.774896 10923 raft_consensus.cc:2932] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:03.775802 10734 catalog_manager.cc:5495] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:03.780898 11012 raft_consensus.cc:2932] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:03.794705 11636 raft_consensus.cc:2932] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:03.804159 10828 tablet_service.cc:1514] Processing DeleteTablet for tablet b6d399a7394c4a59b90a308aedfdf899 with delete_type TABLET_DATA_TOMBSTONED (TS a53441872c0c416b8e2f26f52399f536 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45572
I20260809 15:52:03.853458 11637 raft_consensus.cc:1261] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:03.854782 10733 catalog_manager.cc:5495] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:03.861653 12609 tablet_replica.cc:330] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536: stopping tablet replica
I20260809 15:52:03.866698 11014 raft_consensus.cc:1261] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:52:03.867929 12580 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:03.874228 12609 raft_consensus.cc:2229] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:52:03.875108 12609 raft_consensus.cc:2258] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:52:03.879910 12609 ts_tablet_manager.cc:1905] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:52:03.874286 12573 consensus_queue.cc:1035] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.010s
I20260809 15:52:03.887622 10839 consensus_queue.cc:237] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:03.892269 10847 raft_consensus.cc:1261] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:52:03.893416 12584 consensus_queue.cc:1035] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:52:03.896400 11636 raft_consensus.cc:1261] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:03.897722 12574 consensus_queue.cc:1035] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:03.897709 12609 ts_tablet_manager.cc:1918] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:52:03.900635 12609 log.cc:1196] T b6d399a7394c4a59b90a308aedfdf899 P a53441872c0c416b8e2f26f52399f536: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-0-root/wals/b6d399a7394c4a59b90a308aedfdf899
I20260809 15:52:03.905576 10722 catalog_manager.cc:4841] TS a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): tablet b6d399a7394c4a59b90a308aedfdf899 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:52:03.906500 12573 raft_consensus.cc:2932] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:03.908813 11636 raft_consensus.cc:2932] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
W20260809 15:52:03.925644 10956 debug-util.cc:398] Leaking SignalData structure 0x7b08002fd7c0 after lost signal to thread 9556
I20260809 15:52:03.947113 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:03.949577 10839 raft_consensus.cc:1261] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260809 15:52:03.950112 10956 debug-util.cc:398] Leaking SignalData structure 0x7b080030ed00 after lost signal to thread 10773
I20260809 15:52:03.952320 11012 raft_consensus.cc:1261] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:52:03.952747 12359 consensus_queue.cc:1035] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:03.954586 12585 consensus_queue.cc:1035] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260809 15:52:03.956141 10956 debug-util.cc:398] Leaking SignalData structure 0x7b08001ac180 after lost signal to thread 10875
W20260809 15:52:03.957191 10956 debug-util.cc:398] Leaking SignalData structure 0x7b08000a56e0 after lost signal to thread 10959
W20260809 15:52:03.958190 10956 debug-util.cc:398] Leaking SignalData structure 0x7b0800320380 after lost signal to thread 11047
I20260809 15:52:03.965946 10924 raft_consensus.cc:1261] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:03.971946 11018 raft_consensus.cc:1261] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:52:03.975733 12359 consensus_queue.cc:1035] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260809 15:52:03.977341 10956 debug-util.cc:398] Leaking SignalData structure 0x7b0800341e00 after lost signal to thread 11667
W20260809 15:52:03.977715 10956 debug-util.cc:398] Leaking SignalData structure 0x7b0800316220 after lost signal to thread 12369
I20260809 15:52:03.985754 10924 raft_consensus.cc:1261] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:03.992995 12495 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260809 15:52:03.984251 12580 consensus_queue.cc:1035] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260809 15:52:04.013140 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet e6e09d263f96440a99c99d5a5c6ea477 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:52:04.032691 10923 raft_consensus.cc:2932] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:03.965734 10737 catalog_manager.cc:5495] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 3 to 4, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 4 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:04.049278 10929 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:04.049952 10929 raft_consensus.cc:595] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:04.074151 10840 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:52:04.074761 10840 raft_consensus.cc:595] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:04.082634 11012 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:52:04.083247 11012 raft_consensus.cc:595] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:52:04.086444 10847 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:52:04.087014 10847 raft_consensus.cc:595] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Received request to transfer leadership to TS c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:04.088119 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260809 15:52:04.090515 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:52:04.097520 10931 raft_consensus.cc:1261] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:52:04.109613 12554 consensus_queue.cc:1035] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.013s
W20260809 15:52:04.122550 12611 log.cc:924] Time spent T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.058s user 0.001s sys 0.000s
I20260809 15:52:04.160983 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:04.177486 10734 catalog_manager.cc:5495] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 4 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:04.375993 12433 raft_consensus.cc:979] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536: : Instructing follower c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:52:04.377292 12495 raft_consensus.cc:1067] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Signalling peer c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:52:04.380242 11636 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8"
dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:41482
I20260809 15:52:04.380800 11636 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:04.381160 11636 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:04.429919 11636 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:04.453207 11636 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:04.556442 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:04.557268 11012 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:04.508415 10998 tablet_service.cc:1514] Processing DeleteTablet for tablet e6e09d263f96440a99c99d5a5c6ea477 with delete_type TABLET_DATA_TOMBSTONED (TS d74944fa9fa14ec78b0640ea709ff3aa not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:50526
I20260809 15:52:04.567518 11012 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:04.568089 12492 raft_consensus.cc:979] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
W20260809 15:52:04.590663 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 99622.4us
I20260809 15:52:04.606977 12359 raft_consensus.cc:979] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:04.611174 12359 raft_consensus.cc:1067] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:04.616707 11012 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50542
I20260809 15:52:04.617235 11012 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:04.617580 11012 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
W20260809 15:52:04.640184 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 117618us
I20260809 15:52:04.685561 11012 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:04.688500 12655 tablet_replica.cc:330] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa: stopping tablet replica
I20260809 15:52:04.669920 12554 raft_consensus.cc:1067] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:04.703115 11012 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:04.731801 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 51915us
I20260809 15:52:04.732714 11604 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:04.734764 12657 raft_consensus.cc:2781] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:04.735350 12657 raft_consensus.cc:686] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 3 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:04.736581 12657 consensus_queue.cc:237] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:04.903509 12655 raft_consensus.cc:2229] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 15:52:04.904515 12655 raft_consensus.cc:2258] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:52:04.917479 12655 ts_tablet_manager.cc:1905] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
W20260809 15:52:04.931702 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 239496us
I20260809 15:52:04.942971 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:04.943578 10844 raft_consensus.cc:3032] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:04.943854 10844 raft_consensus.cc:729] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
W20260809 15:52:04.940964 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 91429.2us
I20260809 15:52:04.944541 10844 consensus_queue.cc:260] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:04.950191 11018 raft_consensus.cc:1226] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:04.948140 10844 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
W20260809 15:52:04.972069 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 362670us
I20260809 15:52:04.983253 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:04.983832 10924 raft_consensus.cc:3032] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:04.984186 10924 raft_consensus.cc:729] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:04.987845 10931 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54224
I20260809 15:52:04.984858 10924 consensus_queue.cc:260] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:04.988404 10931 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:04.988749 10931 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:04.988814 10924 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
W20260809 15:52:04.994207 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 319577us
I20260809 15:52:04.994560 10840 consensus_queue.cc:237] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:05.003153 10844 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:05.004206 11018 raft_consensus.cc:1226] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:05.023993 10931 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:05.038012 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:05.038959 10842 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:05.059423 10842 raft_consensus.cc:2454] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:05.061611 10924 raft_consensus.cc:2454] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:05.099849 10927 raft_consensus.cc:1226] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 1. Current term is 2. Ops: []
W20260809 15:52:05.124974 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 89738us
W20260809 15:52:05.140209 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 84797.2us
I20260809 15:52:05.141479 10931 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:05.149199 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:05.149802 11012 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:05.146452 10733 catalog_manager.cc:5495] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 2 to 3, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 3 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:05.159523 11010 raft_consensus.cc:1261] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:52:05.160370 11637 raft_consensus.cc:1261] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:52:05.161793 12660 consensus_queue.cc:1046] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.003s
I20260809 15:52:05.251387 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
W20260809 15:52:05.251806 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 84277.4us
I20260809 15:52:05.265974 11012 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
W20260809 15:52:05.252061 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 84400us
W20260809 15:52:05.270395 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 102968us
I20260809 15:52:05.274631 12660 consensus_queue.cc:1035] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:05.324482 10891 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:05.317112 12626 log.cc:924] Time spent T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.054s user 0.000s sys 0.000s
I20260809 15:52:05.328634 12577 raft_consensus.cc:2781] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:52:05.329113 12577 raft_consensus.cc:686] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:05.329952 12577 consensus_queue.cc:237] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:05.330490 12548 raft_consensus.cc:3032] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:05.331084 12548 raft_consensus.cc:729] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:05.332309 12548 consensus_queue.cc:260] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:05.374933 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 111706us
I20260809 15:52:05.376264 10978 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:05.375981 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 104825us
I20260809 15:52:05.386358 12548 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
W20260809 15:52:05.442956 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 56865.2us
I20260809 15:52:05.445500 12648 raft_consensus.cc:2781] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:52:05.449314 12648 raft_consensus.cc:686] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:05.450532 12648 consensus_queue.cc:237] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:05.453004 11018 raft_consensus.cc:1261] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:05.453775 12655 ts_tablet_manager.cc:1918] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260809 15:52:05.478969 12655 log.cc:1196] T e6e09d263f96440a99c99d5a5c6ea477 P d74944fa9fa14ec78b0640ea709ff3aa: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-2-root/wals/e6e09d263f96440a99c99d5a5c6ea477
I20260809 15:52:05.517851 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:05.531052 10843 raft_consensus.cc:1261] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:05.537701 12656 consensus_queue.cc:1035] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.023s
I20260809 15:52:05.573958 10840 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:05.607544 12527 consensus_queue.cc:1035] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260809 15:52:05.618947 12644 consensus_queue.cc:1035] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260809 15:52:05.659590 12503 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:05.677380 12503 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:05.650677 10719 catalog_manager.cc:4841] TS d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): tablet e6e09d263f96440a99c99d5a5c6ea477 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:52:05.680262 12503 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:05.681346 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 171870us
I20260809 15:52:05.679917 12656 raft_consensus.cc:2932] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:05.683161 11639 raft_consensus.cc:2932] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:05.771401 11010 raft_consensus.cc:1261] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
W20260809 15:52:05.775377 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 73654us
I20260809 15:52:05.777475 12359 consensus_queue.cc:1035] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:52:05.796725 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 369529a0023c4cf4a709bb1a480052b4 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:52:05.796479 10736 catalog_manager.cc:5495] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 1 to 2, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:05.802771 11018 raft_consensus.cc:2932] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:05.839030 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:05.838832 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:05.840876 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:05.842224 10922 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:05.842840 10922 raft_consensus.cc:595] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:05.847821 10840 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "220f58b662044f089f2be802ed24284a"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:52:05.848351 10840 raft_consensus.cc:595] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:05.855290 10840 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:52:05.862300 10840 raft_consensus.cc:595] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Received request to transfer leadership to TS c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:05.870014 10929 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:05.880421 10929 raft_consensus.cc:595] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:05.882567 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260809 15:52:05.884298 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:52:05.895884 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:05.896546 10737 catalog_manager.cc:5495] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 1 to 2, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 2 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:52:05.901130 12359 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:05.901801 12359 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:05.907138 12359 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:05.907806 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:05.909318 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:05.909621 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:05.911324 10891 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:05.913180 12619 raft_consensus.cc:2726] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:05.918427 10978 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:05.919273 12677 raft_consensus.cc:2726] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:05.923480 10841 raft_consensus.cc:1261] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:52:05.925500 12621 consensus_queue.cc:1035] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:05.933813 12621 raft_consensus.cc:979] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:05.934835 12667 raft_consensus.cc:1067] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:05.940972 11012 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50542
I20260809 15:52:05.941550 11012 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:05.941882 11012 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:05.957054 10736 catalog_manager.cc:5495] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 3 to 4, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:05.959268 11012 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:05.962854 10840 raft_consensus.cc:1261] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:52:05.990995 11012 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:05.996304 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:05.996925 10922 raft_consensus.cc:3032] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:05.997282 10922 raft_consensus.cc:729] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:05.998092 10922 consensus_queue.cc:260] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:05.996779 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:05.999518 10840 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:06.000684 10922 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:06.006358 12641 raft_consensus.cc:979] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536: : Instructing follower c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:52:06.006851 12641 raft_consensus.cc:1067] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Signalling peer c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:52:06.008420 11637 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8"
dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:41482
I20260809 15:52:06.008921 11637 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:06.009244 11637 raft_consensus.cc:3037] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:06.009528 10922 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:06.022825 12677 consensus_queue.cc:1035] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:06.029465 12548 raft_consensus.cc:979] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:06.029943 12548 raft_consensus.cc:1067] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:06.031440 10922 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "220f58b662044f089f2be802ed24284a"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54224
I20260809 15:52:06.031973 10922 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:06.032327 10922 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:06.037029 10840 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:06.038849 10978 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:06.039186 10922 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:06.039655 12677 raft_consensus.cc:2781] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:06.040347 12677 raft_consensus.cc:686] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:06.041616 12677 consensus_queue.cc:237] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:06.044003 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:06.044571 10841 raft_consensus.cc:3032] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:06.044855 10841 raft_consensus.cc:729] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:06.045511 10841 consensus_queue.cc:260] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:06.047595 10841 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:06.053613 10841 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:06.054940 10893 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:06.055783 12621 raft_consensus.cc:2781] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:06.056248 12621 raft_consensus.cc:686] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:06.057613 10913 tablet_service.cc:1514] Processing DeleteTablet for tablet 369529a0023c4cf4a709bb1a480052b4 with delete_type TABLET_DATA_TOMBSTONED (TS 20730d75ad954a82b126148f8152a9f3 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:54216
I20260809 15:52:06.058070 12621 consensus_queue.cc:237] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:06.040563 11637 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:06.067271 12697 tablet_replica.cc:330] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3: stopping tablet replica
I20260809 15:52:06.068993 12697 raft_consensus.cc:2229] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:52:06.069962 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:06.070827 10841 raft_consensus.cc:3032] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:06.070978 12667 raft_consensus.cc:979] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:06.071141 10841 raft_consensus.cc:729] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:06.071545 12667 raft_consensus.cc:1067] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:06.071401 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:06.072100 11009 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:06.067291 10922 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:06.074913 11012 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50542
I20260809 15:52:06.075383 11012 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:06.075697 11012 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:06.077513 11637 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:06.078478 10841 consensus_queue.cc:260] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:06.079672 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:06.081008 11015 raft_consensus.cc:3037] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:06.079927 10841 raft_consensus.cc:3037] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:06.094761 11009 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:06.102748 10929 raft_consensus.cc:1261] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:52:06.104851 11012 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:06.069967 12697 raft_consensus.cc:2258] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:52:06.109205 12697 ts_tablet_manager.cc:1905] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:52:06.114384 12666 consensus_queue.cc:1035] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:52:06.125548 11015 raft_consensus.cc:2454] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:52:06.127405 10841 raft_consensus.cc:2454] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:52:06.128595 11604 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:06.128909 11012 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:06.130064 12639 raft_consensus.cc:2781] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:52:06.130564 12639 raft_consensus.cc:686] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:06.131428 12639 consensus_queue.cc:237] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:06.132030 10737 catalog_manager.cc:5495] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 2 to 3, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 3 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:52:06.141302 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:06.141919 10841 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:06.144776 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:06.145581 10929 raft_consensus.cc:3032] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:06.145931 10929 raft_consensus.cc:729] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:06.159137 10841 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:06.161177 10978 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:06.162135 10929 consensus_queue.cc:260] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:06.165323 10736 catalog_manager.cc:5495] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 2 to 3, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 3 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:06.166257 12677 raft_consensus.cc:2781] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:52:06.166757 12677 raft_consensus.cc:686] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:06.167927 12677 consensus_queue.cc:237] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:06.166743 10929 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:06.192684 10929 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:06.201908 10734 catalog_manager.cc:5495] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 1 to 2, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 2 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:06.229918 10737 catalog_manager.cc:5495] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 1 to 2, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 2 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:52:06.238308 12697 ts_tablet_manager.cc:1918] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:52:06.245218 12697 log.cc:1196] T 369529a0023c4cf4a709bb1a480052b4 P 20730d75ad954a82b126148f8152a9f3: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-1-root/wals/369529a0023c4cf4a709bb1a480052b4
I20260809 15:52:06.247144 10722 catalog_manager.cc:4841] TS 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): tablet 369529a0023c4cf4a709bb1a480052b4 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:52:06.464433 11015 raft_consensus.cc:1261] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:52:06.465835 12621 consensus_queue.cc:1035] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:06.475975 10838 raft_consensus.cc:1261] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:52:06.487375 12621 consensus_queue.cc:1035] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:06.511845 10929 raft_consensus.cc:1261] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:52:06.513319 12666 consensus_queue.cc:1035] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:06.536650 10838 raft_consensus.cc:1261] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:52:06.539356 12666 consensus_queue.cc:1035] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:06.601286 10837 raft_consensus.cc:1261] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:06.602669 12639 consensus_queue.cc:1035] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:06.620888 11015 raft_consensus.cc:1261] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:06.622937 12639 consensus_queue.cc:1035] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:06.649287 10837 raft_consensus.cc:1261] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:52:06.650791 12677 consensus_queue.cc:1035] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:06.663077 10929 raft_consensus.cc:1261] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:52:06.667361 12678 consensus_queue.cc:1035] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:06.901885 11015 consensus_queue.cc:237] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:06.923894 10929 raft_consensus.cc:1261] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:52:06.927033 12678 consensus_queue.cc:1035] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260809 15:52:06.940328 10978 consensus_peers.cc:487] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7c8eaf50bcb3457bb4b1e30406cc349f. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:52:06.948763 10838 raft_consensus.cc:1261] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:52:06.953084 12678 consensus_queue.cc:1035] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:06.961380 12682 raft_consensus.cc:2932] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:06.965308 10929 raft_consensus.cc:2932] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:06.968393 10841 raft_consensus.cc:2932] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:07.031513 10838 consensus_queue.cc:237] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:07.032150 10736 catalog_manager.cc:5495] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:07.054896 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
W20260809 15:52:07.104660 10811 consensus_peers.cc:487] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 0b2b0df042e44725a7c91deae3df1956. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:52:07.110456 11015 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260809 15:52:07.113528 10929 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260809 15:52:07.114660 12641 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:07.116793 12641 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:07.130028 12641 raft_consensus.cc:2932] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:07.135653 10927 raft_consensus.cc:2932] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:07.148350 11009 raft_consensus.cc:2932] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:07.159017 10734 catalog_manager.cc:5495] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:07.163229 10927 consensus_queue.cc:237] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:07.171844 11015 raft_consensus.cc:1261] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260809 15:52:07.172361 10838 raft_consensus.cc:1261] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260809 15:52:07.174054 12667 consensus_queue.cc:1035] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260809 15:52:07.175050 10893 consensus_peers.cc:487] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: e5757bc9ac9441c3bdfa6c5b2b743b6d. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:52:07.176087 12667 consensus_queue.cc:1035] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:07.187292 11015 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:52:07.187865 11015 raft_consensus.cc:595] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:07.190310 11012 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:52:07.190781 11012 raft_consensus.cc:595] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:52:07.198307 11009 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:52:07.198928 11009 raft_consensus.cc:595] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:52:07.198848 12621 raft_consensus.cc:2932] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:07.200660 11012 raft_consensus.cc:2932] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:07.202235 10838 raft_consensus.cc:2932] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:07.214854 12640 raft_consensus.cc:979] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:07.215343 12640 raft_consensus.cc:1067] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:07.224768 10927 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:07.225310 10927 raft_consensus.cc:595] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:52:07.230178 10841 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45578
I20260809 15:52:07.230278 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 4
I20260809 15:52:07.230655 10841 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:07.231001 10841 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:07.232106 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:52:07.231419 10734 catalog_manager.cc:5495] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:07.232810 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:07.244177 10929 consensus_queue.cc:237] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:07.250396 10841 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:07.254348 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:07.254907 11637 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:07.256150 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:07.257061 11009 raft_consensus.cc:3032] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:07.257364 11009 raft_consensus.cc:729] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:07.258129 11009 consensus_queue.cc:260] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:07.261045 11009 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:07.259009 10841 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:07.262260 11637 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:07.264571 10811 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:07.265362 12633 raft_consensus.cc:2781] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:07.265784 12633 raft_consensus.cc:686] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:07.266613 12633 consensus_queue.cc:237] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:07.281018 11015 raft_consensus.cc:1261] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:52:07.282490 12359 consensus_queue.cc:1035] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260809 15:52:07.283468 10893 consensus_peers.cc:487] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 974451500a6e434f8d407d9f178ab710. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:52:07.291837 12640 raft_consensus.cc:979] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:07.292321 12640 raft_consensus.cc:1067] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:07.294713 10837 raft_consensus.cc:1261] T 974451500a6e434f8d407d9f178ab710 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:52:07.296031 12359 raft_consensus.cc:2932] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:07.297483 11015 raft_consensus.cc:2932] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:07.299885 10929 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54236
I20260809 15:52:07.300452 10929 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:07.300756 10929 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:07.302587 12667 consensus_queue.cc:1035] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:07.336328 10733 catalog_manager.cc:5495] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 2 to 3, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 3 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:07.328838 11009 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:07.346563 10929 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:07.348760 10929 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:07.360931 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:07.361464 11012 raft_consensus.cc:3032] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:07.361701 11012 raft_consensus.cc:729] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:07.362447 11012 consensus_queue.cc:260] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:07.363477 11012 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:07.370007 11012 raft_consensus.cc:2454] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:07.370970 10891 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:07.372730 10736 catalog_manager.cc:5495] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:07.371968 12729 raft_consensus.cc:2781] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:07.383234 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:07.383797 11637 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:07.382211 12729 raft_consensus.cc:686] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:07.390345 11637 raft_consensus.cc:2454] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:07.389155 12729 consensus_queue.cc:237] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:07.414408 10837 raft_consensus.cc:2932] T 974451500a6e434f8d407d9f178ab710 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:07.448414 12621 raft_consensus.cc:979] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:07.448884 12621 raft_consensus.cc:1067] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:07.453372 10841 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45594
I20260809 15:52:07.459103 10841 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:07.459357 12734 ts_tablet_manager.cc:927] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:07.459491 10841 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:07.465725 12734 tablet_copy_client.cc:316] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.3:39223
I20260809 15:52:07.464890 10736 catalog_manager.cc:5495] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 3 to 4, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 4 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:07.467249 11027 tablet_copy_service.cc:140] P d74944fa9fa14ec78b0640ea709ff3aa: Received BeginTabletCopySession request for tablet 7c8eaf50bcb3457bb4b1e30406cc349f from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:39978)
I20260809 15:52:07.467782 11027 tablet_copy_service.cc:161] P d74944fa9fa14ec78b0640ea709ff3aa: Beginning new tablet copy session on tablet 7c8eaf50bcb3457bb4b1e30406cc349f from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:39978: session id = c7aa762318cb4e97875333f10679a9f9-7c8eaf50bcb3457bb4b1e30406cc349f
I20260809 15:52:07.476490 10841 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:07.478258 11027 tablet_copy_source_session.cc:205] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:52:07.479019 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:07.485570 10929 raft_consensus.cc:3032] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:07.485875 10929 raft_consensus.cc:729] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:07.486763 10929 consensus_queue.cc:260] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:07.487803 10929 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:07.495684 10929 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:07.496986 10811 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:07.497782 10841 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:07.483492 12734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c8eaf50bcb3457bb4b1e30406cc349f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:52:07.498476 12641 raft_consensus.cc:2781] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:07.500964 12641 raft_consensus.cc:686] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:07.501926 12641 consensus_queue.cc:237] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:07.509812 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:07.510607 11009 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:07.542555 12737 ts_tablet_manager.cc:927] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615)
I20260809 15:52:07.550776 11009 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:07.551399 10736 catalog_manager.cc:5495] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 2 to 3, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 3 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:07.569536 12734 tablet_copy_client.cc:791] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:52:07.570420 12734 tablet_copy_client.cc:655] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:52:07.578759 12737 tablet_copy_client.cc:316] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.1:46615
I20260809 15:52:07.580269 10858 tablet_copy_service.cc:140] P a53441872c0c416b8e2f26f52399f536: Received BeginTabletCopySession request for tablet 0b2b0df042e44725a7c91deae3df1956 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:54342)
I20260809 15:52:07.580771 10858 tablet_copy_service.cc:161] P a53441872c0c416b8e2f26f52399f536: Beginning new tablet copy session on tablet 0b2b0df042e44725a7c91deae3df1956 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:54342: session id = c7aa762318cb4e97875333f10679a9f9-0b2b0df042e44725a7c91deae3df1956
I20260809 15:52:07.587582 12734 tablet_copy_client.cc:531] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:52:07.593708 10858 tablet_copy_source_session.cc:205] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:52:07.604231 12734 tablet_bootstrap.cc:492] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:52:07.607806 12737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b2b0df042e44725a7c91deae3df1956. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:52:07.624431 12737 tablet_copy_client.cc:791] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:52:07.624985 12737 tablet_copy_client.cc:655] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:52:07.637169 12737 tablet_copy_client.cc:531] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:52:07.650369 12737 tablet_bootstrap.cc:492] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:52:07.650627 12734 tablet_bootstrap.cc:492] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:52:07.651531 12734 tablet_bootstrap.cc:492] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:52:07.652201 12734 ts_tablet_manager.cc:1397] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.048s user 0.020s sys 0.002s
I20260809 15:52:07.655146 12734 raft_consensus.cc:348] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:07.656301 12734 raft_consensus.cc:729] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:52:07.659801 12734 consensus_queue.cc:260] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:07.663345 12734 ts_tablet_manager.cc:1428] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s
I20260809 15:52:07.671442 12740 ts_tablet_manager.cc:927] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:07.674256 11027 tablet_copy_service.cc:342] P d74944fa9fa14ec78b0640ea709ff3aa: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-7c8eaf50bcb3457bb4b1e30406cc349f received from {username='slave'} at 127.0.0.1:39978
I20260809 15:52:07.674827 11027 tablet_copy_service.cc:433] P d74944fa9fa14ec78b0640ea709ff3aa: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-7c8eaf50bcb3457bb4b1e30406cc349f on tablet 7c8eaf50bcb3457bb4b1e30406cc349f with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:07.682574 12737 tablet_bootstrap.cc:492] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:52:07.683871 12737 tablet_bootstrap.cc:492] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:52:07.684799 12737 ts_tablet_manager.cc:1397] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.035s user 0.024s sys 0.003s
I20260809 15:52:07.692965 12737 raft_consensus.cc:348] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:07.694273 12737 raft_consensus.cc:729] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:52:07.696867 12737 consensus_queue.cc:260] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:07.699961 12737 ts_tablet_manager.cc:1428] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.015s user 0.006s sys 0.008s
I20260809 15:52:07.701452 10858 tablet_copy_service.cc:342] P a53441872c0c416b8e2f26f52399f536: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-0b2b0df042e44725a7c91deae3df1956 received from {username='slave'} at 127.0.0.1:54342
I20260809 15:52:07.701896 10858 tablet_copy_service.cc:433] P a53441872c0c416b8e2f26f52399f536: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-0b2b0df042e44725a7c91deae3df1956 on tablet 0b2b0df042e44725a7c91deae3df1956 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:07.703792 11010 raft_consensus.cc:1261] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:07.708052 12548 consensus_queue.cc:1035] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:07.720233 12740 tablet_copy_client.cc:316] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.2:32783
I20260809 15:52:07.730052 10942 tablet_copy_service.cc:140] P 20730d75ad954a82b126148f8152a9f3: Received BeginTabletCopySession request for tablet e5757bc9ac9441c3bdfa6c5b2b743b6d from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:52758)
I20260809 15:52:07.730706 10942 tablet_copy_service.cc:161] P 20730d75ad954a82b126148f8152a9f3: Beginning new tablet copy session on tablet e5757bc9ac9441c3bdfa6c5b2b743b6d from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:52758: session id = c7aa762318cb4e97875333f10679a9f9-e5757bc9ac9441c3bdfa6c5b2b743b6d
I20260809 15:52:07.738698 11637 raft_consensus.cc:1261] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:07.740916 12548 consensus_queue.cc:1035] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:07.745949 10942 tablet_copy_source_session.cc:205] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:52:07.761236 12740 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5757bc9ac9441c3bdfa6c5b2b743b6d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:52:07.788784 12677 raft_consensus.cc:979] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:07.789660 12677 raft_consensus.cc:1067] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:07.792133 10837 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45578
I20260809 15:52:07.792893 10837 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:07.793210 10837 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:07.838346 11010 raft_consensus.cc:1261] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260809 15:52:07.838496 10837 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:07.842242 10837 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:07.842571 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:07.843216 10927 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:07.848289 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:07.848950 11009 raft_consensus.cc:3032] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:07.849345 12359 consensus_queue.cc:1035] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 15:52:07.849426 11009 raft_consensus.cc:729] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:07.850783 11009 consensus_queue.cc:260] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:07.852123 11009 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:07.858485 12740 tablet_copy_client.cc:791] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:52:07.859146 12740 tablet_copy_client.cc:655] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:52:07.891731 12740 tablet_copy_client.cc:531] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:52:07.897776 11639 raft_consensus.cc:1203] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:52:07.899706 10927 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:07.909174 12737 ts_tablet_manager.cc:927] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:07.917801 10811 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:07.923504 12633 raft_consensus.cc:2781] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election won for term 4
W20260809 15:52:07.992906 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 50895.5us
I20260809 15:52:07.974277 12737 tablet_copy_client.cc:316] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.2:32783
I20260809 15:52:08.002020 11639 raft_consensus.cc:1261] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260809 15:52:08.018071 12621 consensus_queue.cc:1035] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260809 15:52:08.021190 11009 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:08.034921 10942 tablet_copy_service.cc:140] P 20730d75ad954a82b126148f8152a9f3: Received BeginTabletCopySession request for tablet 974451500a6e434f8d407d9f178ab710 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:52758)
I20260809 15:52:08.035876 10942 tablet_copy_service.cc:161] P 20730d75ad954a82b126148f8152a9f3: Beginning new tablet copy session on tablet 974451500a6e434f8d407d9f178ab710 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:52758: session id = c7aa762318cb4e97875333f10679a9f9-974451500a6e434f8d407d9f178ab710
I20260809 15:52:08.037777 11015 raft_consensus.cc:1261] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:52:08.026150 12633 raft_consensus.cc:686] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
W20260809 15:52:08.141242 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 96498.8us
I20260809 15:52:08.177379 12757 consensus_queue.cc:1035] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:08.178889 12633 consensus_queue.cc:237] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:08.263506 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 184002us
W20260809 15:52:08.315716 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 172853us
I20260809 15:52:08.343871 10929 raft_consensus.cc:1261] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:52:08.348430 12776 consensus_queue.cc:1035] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:52:08.365710 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:08.379413 12740 tablet_bootstrap.cc:492] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:52:08.389513 10942 tablet_copy_source_session.cc:205] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3: Tablet Copy: opened 0 blocks and 1 log segments
W20260809 15:52:08.413343 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 186068us
W20260809 15:52:08.437675 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 104286us
I20260809 15:52:08.456920 10736 catalog_manager.cc:5495] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 3 to 4, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 4 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:08.479322 11637 raft_consensus.cc:1203] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
W20260809 15:52:08.590742 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 56355.7us
W20260809 15:52:08.590615 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 56194.2us
W20260809 15:52:08.615060 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 79502.9us
I20260809 15:52:08.653553 12737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 974451500a6e434f8d407d9f178ab710. 1 dirs total, 0 dirs full, 0 dirs failed
W20260809 15:52:08.718896 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 61454.7us
W20260809 15:52:08.738752 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 44465us
W20260809 15:52:08.831918 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 88204.7us
W20260809 15:52:08.851924 12736 log.cc:924] Time spent T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.199s user 0.012s sys 0.000s
I20260809 15:52:08.861964 12740 tablet_bootstrap.cc:492] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:52:08.863426 12740 tablet_bootstrap.cc:492] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:52:08.863441 12737 tablet_copy_client.cc:791] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:52:08.864239 12740 ts_tablet_manager.cc:1397] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.485s user 0.020s sys 0.022s
I20260809 15:52:08.864465 12737 tablet_copy_client.cc:655] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
W20260809 15:52:08.904579 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 144776us
I20260809 15:52:08.868952 12740 raft_consensus.cc:348] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:08.937855 12740 raft_consensus.cc:729] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:52:08.939687 12740 consensus_queue.cc:260] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:08.967542 12785 raft_consensus.cc:1050] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:52:08.983301 11009 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:52:08.987444 11009 raft_consensus.cc:595] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:09.041250 11010 raft_consensus.cc:1261] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:52:09.056565 11636 raft_consensus.cc:1203] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20260809 15:52:09.061234 12761 raft_consensus.cc:979] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:09.061949 12754 raft_consensus.cc:1067] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:09.076494 12740 ts_tablet_manager.cc:1428] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.212s user 0.009s sys 0.003s
I20260809 15:52:09.077621 10923 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54236
I20260809 15:52:09.083581 10923 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:09.084121 10923 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:09.089720 12737 tablet_copy_client.cc:531] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:52:09.079417 12785 consensus_queue.cc:237] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:09.080338 12792 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
W20260809 15:52:09.105756 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 288947us
I20260809 15:52:09.106513 10942 tablet_copy_service.cc:342] P 20730d75ad954a82b126148f8152a9f3: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-e5757bc9ac9441c3bdfa6c5b2b743b6d received from {username='slave'} at 127.0.0.1:52758
I20260809 15:52:09.107127 10942 tablet_copy_service.cc:433] P 20730d75ad954a82b126148f8152a9f3: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-e5757bc9ac9441c3bdfa6c5b2b743b6d on tablet e5757bc9ac9441c3bdfa6c5b2b743b6d with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:09.152604 12737 tablet_bootstrap.cc:492] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:52:09.166884 10929 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:09.167596 10929 raft_consensus.cc:595] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:52:09.169832 10929 raft_consensus.cc:1261] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:52:09.173472 10923 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:09.193485 10923 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:09.204241 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:09.204972 10839 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:09.242300 10839 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:09.258289 10893 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:09.264249 10926 raft_consensus.cc:1261] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:09.265313 11639 raft_consensus.cc:1261] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:09.264323 10848 raft_consensus.cc:1261] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:09.267663 12786 raft_consensus.cc:2781] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:09.268200 12786 raft_consensus.cc:686] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:09.268738 12812 consensus_queue.cc:1035] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:52:09.269258 12786 consensus_queue.cc:237] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:09.274843 12811 consensus_queue.cc:1035] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:52:09.278545 12763 raft_consensus.cc:1050] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:52:09.281757 12763 consensus_queue.cc:237] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:09.289140 12811 consensus_queue.cc:1035] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:09.305478 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260809 15:52:09.306671 12780 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:09.312860 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:52:09.314595 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:09.318015 11016 raft_consensus.cc:3032] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:09.319887 11016 raft_consensus.cc:729] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:09.318841 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:09.444458 11016 consensus_queue.cc:260] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:09.446226 11016 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
W20260809 15:52:09.489069 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 62064.4us
I20260809 15:52:09.528769 11016 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
W20260809 15:52:09.621729 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 64155.9us
W20260809 15:52:09.624866 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 69557.8us
W20260809 15:52:09.658488 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 71103.5us
I20260809 15:52:09.660198 12769 raft_consensus.cc:979] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:09.661300 12769 raft_consensus.cc:1067] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
W20260809 15:52:09.703976 12822 log.cc:924] Time spent T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.061s user 0.000s sys 0.002s
I20260809 15:52:09.717705 12770 raft_consensus.cc:2932] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:09.722743 10839 raft_consensus.cc:2932] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
W20260809 15:52:09.733736 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 83115.1us
W20260809 15:52:09.735554 12819 log.cc:924] Time spent T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.183s user 0.002s sys 0.000s
I20260809 15:52:09.742859 10848 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45594
I20260809 15:52:09.743419 10848 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:09.746220 10848 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:09.752543 10848 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:09.754504 10848 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:09.755452 11639 raft_consensus.cc:2932] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:09.757216 12737 tablet_bootstrap.cc:492] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:52:09.758183 12737 tablet_bootstrap.cc:492] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:52:09.758878 12737 ts_tablet_manager.cc:1397] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.607s user 0.045s sys 0.019s
I20260809 15:52:09.759935 11014 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:52:09.761508 12774 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:09.767016 11641 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:52:09.767532 10734 catalog_manager.cc:5495] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 2 to 3, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 3 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
W20260809 15:52:09.797818 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 84898.9us
I20260809 15:52:09.794667 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:09.799609 11636 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:09.801888 12790 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:09.805930 10931 raft_consensus.cc:2932] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:09.817276 10923 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:52:09.827019 12773 raft_consensus.cc:1050] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:52:09.826387 10736 catalog_manager.cc:5495] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:09.830192 12773 consensus_queue.cc:237] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:09.839210 10848 raft_consensus.cc:1261] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:52:09.840078 11636 raft_consensus.cc:2454] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:09.764212 12737 raft_consensus.cc:348] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:09.851830 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:09.852365 10929 raft_consensus.cc:3032] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:09.852710 10929 raft_consensus.cc:729] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:09.853497 10929 consensus_queue.cc:260] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:09.854651 10929 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:09.861217 10811 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:09.862336 12781 raft_consensus.cc:2781] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:09.862808 12781 raft_consensus.cc:686] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:09.863615 12781 consensus_queue.cc:237] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:09.871879 12831 consensus_queue.cc:1035] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.003s
I20260809 15:52:09.881084 10929 raft_consensus.cc:2454] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:09.850817 12737 raft_consensus.cc:729] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:52:09.902184 12737 consensus_queue.cc:260] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:09.905592 12737 ts_tablet_manager.cc:1428] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.146s user 0.009s sys 0.000s
I20260809 15:52:09.915742 12780 raft_consensus.cc:2932] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:09.919857 11014 raft_consensus.cc:2932] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:09.929195 10942 tablet_copy_service.cc:342] P 20730d75ad954a82b126148f8152a9f3: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-974451500a6e434f8d407d9f178ab710 received from {username='slave'} at 127.0.0.1:52758
I20260809 15:52:09.929623 10942 tablet_copy_service.cc:433] P 20730d75ad954a82b126148f8152a9f3: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-974451500a6e434f8d407d9f178ab710 on tablet 974451500a6e434f8d407d9f178ab710 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:09.935711 12790 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260809 15:52:09.948355 11009 raft_consensus.cc:1261] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:52:09.951712 11639 raft_consensus.cc:1261] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:52:09.952823 10837 raft_consensus.cc:1261] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:52:09.960078 12773 consensus_queue.cc:1035] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:52:09.961911 12773 consensus_queue.cc:1035] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:52:09.970556 11641 raft_consensus.cc:2932] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:09.983210 11009 consensus_queue.cc:237] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:10.092598 11639 raft_consensus.cc:1261] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:52:10.221061 11642 raft_consensus.cc:1203] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
W20260809 15:52:10.310631 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 216950us
I20260809 15:52:10.313437 12678 consensus_queue.cc:1035] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:10.436587 10927 raft_consensus.cc:1261] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:52:10.451122 10923 raft_consensus.cc:2932] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:10.502794 11014 raft_consensus.cc:1261] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:52:10.530699 12770 raft_consensus.cc:2932] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
W20260809 15:52:10.677351 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 357414us
I20260809 15:52:10.785490 12826 consensus_queue.cc:1035] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:52:10.790136 12826 consensus_queue.cc:1035] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:10.995580 12678 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:10.996356 12678 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:11.035561 12812 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:11.036079 12812 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:11.062613 10737 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59504 (request call id 209) took 1077 ms. Trace:
W20260809 15:52:11.068953 10976 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.561s user 0.006s sys 0.022s
I20260809 15:52:11.052961 12855 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:52:11.131567 10976 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.624s user 0.006s sys 0.022s
I20260809 15:52:11.038152 12812 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:11.131934 12855 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:11.060421 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:11.134160 12855 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:11.127041 10737 rpcz_store.cc:276] 0809 15:52:09.985330 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:09.986600 (+ 1270us) service_pool.cc:227] Handling call
0809 15:52:11.062399 (+1075799us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":1}
W20260809 15:52:11.072248 10837 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.088s user 0.027s sys 0.019s
W20260809 15:52:11.135569 10837 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.151s user 0.028s sys 0.020s
I20260809 15:52:11.113462 10733 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33054 (request call id 337) took 1118 ms. Trace:
I20260809 15:52:11.116647 12785 consensus_queue.cc:1035] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.015s
I20260809 15:52:11.100556 12678 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:11.037307 12770 consensus_meta.cc:287] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.505s user 0.006s sys 0.000s
I20260809 15:52:11.143289 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:11.067008 10929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:54216 (request call id 1305) took 1140 ms. Trace:
I20260809 15:52:11.137802 10837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 1726) took 1155 ms. Trace:
I20260809 15:52:11.136286 10733 rpcz_store.cc:276] 0809 15:52:09.990323 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:09.991365 (+ 1042us) service_pool.cc:227] Handling call
0809 15:52:11.108631 (+1117266us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5,"reported_tablets":5}
I20260809 15:52:11.145112 10929 rpcz_store.cc:276] 0809 15:52:09.926284 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:09.935527 (+ 9243us) service_pool.cc:227] Handling call
0809 15:52:11.066326 (+1130799us) spinlock_profiling.cc:243] Waited 23.7 ms on lock 0x7b540024c7d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478af0 00007f79694fbf2a 00007f796e438c6e 00007f7967b2f807 00007f7967b2f792 00007f7967b2f6c1 00007f7967b2f63c 00007f7967b2e8f2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:11.066900 (+ 574us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":54592768}
I20260809 15:52:11.094458 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:11.160779 12855 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:11.161582 12855 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:11.100937 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:11.169951 12855 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:11.151757 10837 rpcz_store.cc:276] 0809 15:52:09.982391 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:09.982536 (+ 145us) service_pool.cc:227] Handling call
0809 15:52:09.983112 (+ 576us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:09.983484 (+ 372us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:09.983519 (+ 35us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:09.984387 (+ 868us) log.cc:842] Serialized 540 byte log entry
0809 15:52:11.137446 (+1153059us) raft_consensus.cc:1631] Marking committed up to 3
0809 15:52:11.137497 (+ 51us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:11.137566 (+ 69us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:11.137607 (+ 41us) raft_consensus.cc:1671] finished
0809 15:52:11.137616 (+ 9us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:11.137729 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":1151165,"threads_started":1,"wal-append.queue_time_us":544205}
I20260809 15:52:11.197162 12862 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:11.195127 10923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1491) took 1231 ms. Trace:
I20260809 15:52:11.198211 12862 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:11.199925 10923 rpcz_store.cc:276] 0809 15:52:09.962962 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:09.963081 (+ 119us) service_pool.cc:227] Handling call
0809 15:52:09.963591 (+ 510us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:09.963985 (+ 394us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:09.964028 (+ 43us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:09.964965 (+ 937us) log.cc:842] Serialized 540 byte log entry
0809 15:52:10.450985 (+486020us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:52:11.193966 (+742981us) log.cc:842] Serialized 14 byte log entry
0809 15:52:11.194210 (+ 244us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:11.194260 (+ 50us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:11.194285 (+ 25us) raft_consensus.cc:1671] finished
0809 15:52:11.194291 (+ 6us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:11.194361 (+ 70us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":508,"raft.run_cpu_time_us":117,"raft.run_wall_time_us":117,"thread_start_us":485668,"threads_started":1,"wal-append.queue_time_us":483647}
I20260809 15:52:11.205757 12862 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:11.213824 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 65992us
I20260809 15:52:11.226958 12855 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:11.227667 12855 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:11.149008 11636 raft_consensus.cc:2932] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:11.235260 12876 raft_consensus.cc:2932] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:11.250455 12855 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:11.273185 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:11.283644 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:11.286207 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:11.286525 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:11.288516 10923 raft_consensus.cc:1261] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260809 15:52:11.289914 12880 consensus_queue.cc:1035] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260809 15:52:11.298534 10847 raft_consensus.cc:2932] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:11.314990 12805 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:11.315554 12805 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:11.317798 12805 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:11.319329 10736 catalog_manager.cc:5495] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 3 to 4, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 4 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:11.320815 10736 catalog_manager.cc:5495] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:11.330021 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:11.333293 11637 raft_consensus.cc:1261] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260809 15:52:11.335618 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:11.340538 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:11.342733 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:11.349220 10811 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:11.352257 12880 raft_consensus.cc:2726] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:11.353742 10811 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:11.354350 12857 consensus_queue.cc:1035] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260809 15:52:11.356245 12869 raft_consensus.cc:2726] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:11.362335 12866 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:11.362919 12866 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:11.367746 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:11.368366 10847 raft_consensus.cc:2454] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:52:11.369479 12866 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:11.381979 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:11.383438 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:11.384582 12853 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:11.384721 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:11.385080 12853 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:11.386960 12853 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:11.377732 12852 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:11.388594 12852 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:11.389053 12770 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:50526 (request call id 1070) took 1411 ms. Trace:
I20260809 15:52:11.389480 12770 rpcz_store.cc:276] 0809 15:52:09.978190 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:09.980821 (+ 2631us) service_pool.cc:227] Handling call
0809 15:52:09.998672 (+ 17851us) log.cc:842] Serialized 480 byte log entry
0809 15:52:11.388973 (+1390301us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":79612,"raft.run_cpu_time_us":1155,"raft.run_wall_time_us":1153,"spinlock_wait_cycles":16128}
I20260809 15:52:11.390601 12852 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:11.391404 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:11.373903 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 227624us
I20260809 15:52:11.396190 10891 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:11.396644 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:11.397820 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:11.398537 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:11.401062 12874 raft_consensus.cc:2726] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:11.402568 11637 raft_consensus.cc:2932] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:11.405467 10809 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:11.406104 10719 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 7c8eaf50bcb3457bb4b1e30406cc349f with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:52:11.406236 12869 raft_consensus.cc:2726] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260809 15:52:11.410686 12885 log.cc:924] Time spent T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.103s user 0.003s sys 0.000s
I20260809 15:52:11.414361 11606 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:11.415091 12805 raft_consensus.cc:2726] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:11.416599 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:11.419253 12767 raft_consensus.cc:482] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:11.419757 12767 raft_consensus.cc:504] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:11.421840 12767 leader_election.cc:288] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:11.427030 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:11.428478 10893 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:11.433373 12770 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:11.433930 12770 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:11.436322 12770 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:11.447132 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:11.450793 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:11.458979 10978 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:11.462303 12785 raft_consensus.cc:2726] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:11.463395 10978 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:11.466250 12785 raft_consensus.cc:2726] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:11.468066 12875 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:11.467770 12731 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:11.468693 12875 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:11.469309 12731 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:11.469594 10733 catalog_manager.cc:5495] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:11.472172 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:11.472373 12731 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:11.475216 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:11.475273 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:11.475883 11635 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:11.475956 11015 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:11.478999 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:11.479846 10891 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:11.488032 10978 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:11.489027 12875 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:11.464216 11606 leader_election.cc:302] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:11.492237 12891 raft_consensus.cc:2726] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:11.494931 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:11.498903 12805 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:11.499396 12805 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:11.501554 12805 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:11.502130 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:11.502275 10923 consensus_queue.cc:237] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:11.504930 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:11.506631 10893 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:11.507411 12731 raft_consensus.cc:2726] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:11.509884 10978 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:11.511618 12785 raft_consensus.cc:2726] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:11.479625 10931 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:11.515250 12876 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:11.517062 12876 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:11.517508 12876 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:11.517581 12678 raft_consensus.cc:2726] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:11.515982 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:11.518575 10927 raft_consensus.cc:2932] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:11.524057 11637 raft_consensus.cc:1261] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:11.528044 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:11.529142 12876 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:11.535789 10736 catalog_manager.cc:5495] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New cstate: current_term: 1 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:11.547880 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:11.548499 11636 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:11.538048 11606 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:11.540035 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:11.550350 12891 raft_consensus.cc:2726] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:11.550604 11013 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:11.552418 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:11.554917 10847 raft_consensus.cc:1261] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:11.544185 12876 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:11.545248 12878 consensus_queue.cc:1035] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.013s
I20260809 15:52:11.558354 10978 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:11.544386 12855 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:11.559350 12855 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:11.561262 12855 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:11.561195 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:11.561827 11637 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:11.564467 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:11.567440 12872 raft_consensus.cc:2726] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:11.571195 11013 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:11.571753 12678 raft_consensus.cc:2741] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260809 15:52:11.577992 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:11.587060 11637 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:11.587239 10828 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c8eaf50bcb3457bb4b1e30406cc349f with delete_type TABLET_DATA_TOMBSTONED (TS a53441872c0c416b8e2f26f52399f536 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45572
I20260809 15:52:11.597296 10811 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:11.599244 12856 raft_consensus.cc:2726] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:11.599913 12878 consensus_queue.cc:1035] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:11.600629 12900 tablet_replica.cc:330] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536: stopping tablet replica
I20260809 15:52:11.602635 11013 raft_consensus.cc:1226] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:11.607858 12855 consensus_queue.cc:1046] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:11.608358 12878 raft_consensus.cc:2932] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:11.608744 12855 raft_consensus.cc:3032] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:11.610350 12855 raft_consensus.cc:729] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:11.610448 12900 raft_consensus.cc:2229] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:52:11.611384 12900 raft_consensus.cc:2258] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:52:11.611349 12855 consensus_queue.cc:260] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:11.613112 12855 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:11.617208 12900 ts_tablet_manager.cc:1905] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:52:11.619230 10893 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:11.620035 12731 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:11.624568 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet e5757bc9ac9441c3bdfa6c5b2b743b6d with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:52:11.620744 12731 raft_consensus.cc:686] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:11.624220 11637 raft_consensus.cc:2932] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:11.626514 12731 consensus_queue.cc:237] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:11.637014 10736 catalog_manager.cc:5495] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 4 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:11.640156 10843 raft_consensus.cc:2932] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:11.643072 12872 raft_consensus.cc:1050] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:52:11.644232 12900 ts_tablet_manager.cc:1918] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:52:11.645609 12872 consensus_queue.cc:237] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:11.648051 10998 tablet_service.cc:1514] Processing DeleteTablet for tablet e5757bc9ac9441c3bdfa6c5b2b743b6d with delete_type TABLET_DATA_TOMBSTONED (TS d74944fa9fa14ec78b0640ea709ff3aa not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:50526
I20260809 15:52:11.644858 12900 log.cc:1196] T 7c8eaf50bcb3457bb4b1e30406cc349f P a53441872c0c416b8e2f26f52399f536: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-0-root/wals/7c8eaf50bcb3457bb4b1e30406cc349f
I20260809 15:52:11.657809 10722 catalog_manager.cc:4841] TS a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): tablet 7c8eaf50bcb3457bb4b1e30406cc349f (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:52:11.667567 11013 raft_consensus.cc:1261] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:11.683037 11637 raft_consensus.cc:1261] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:11.679919 12902 tablet_replica.cc:330] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa: stopping tablet replica
I20260809 15:52:11.691288 10736 catalog_manager.cc:5495] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 2 to 3, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 3 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:11.685703 12872 consensus_queue.cc:1035] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260809 15:52:11.701985 12878 consensus_queue.cc:1035] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:11.702356 10843 raft_consensus.cc:1261] T 974451500a6e434f8d407d9f178ab710 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:11.708016 12731 consensus_queue.cc:1035] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:11.723197 12902 raft_consensus.cc:2229] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 15:52:11.724061 12902 pending_rounds.cc:62] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa: Trying to abort 1 pending ops.
I20260809 15:52:11.726720 12878 raft_consensus.cc:2932] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:11.726189 12902 pending_rounds.cc:68] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa: Aborting op as it isn't in flight: id { term: 2 index: 4 } timestamp: 7316646829375733760 op_type: CHANGE_CONFIG_OP change_config_record { tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" old_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } } new_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } } }
I20260809 15:52:11.728947 12902 raft_consensus.cc:2894] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Aborting config change with OpId 2.4: Aborted: Op aborted
I20260809 15:52:11.730922 12902 raft_consensus.cc:2866] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted
I20260809 15:52:11.731329 12902 raft_consensus.cc:2258] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:52:11.733246 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:11.733292 11637 raft_consensus.cc:2932] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:11.741098 10837 raft_consensus.cc:2932] T 974451500a6e434f8d407d9f178ab710 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:11.745935 11013 raft_consensus.cc:2932] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:11.751971 10736 catalog_manager.cc:5495] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:11.756541 12902 ts_tablet_manager.cc:1905] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:52:11.777472 12902 ts_tablet_manager.cc:1918] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260809 15:52:11.777967 12902 log.cc:1196] T e5757bc9ac9441c3bdfa6c5b2b743b6d P d74944fa9fa14ec78b0640ea709ff3aa: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-2-root/wals/e5757bc9ac9441c3bdfa6c5b2b743b6d
I20260809 15:52:11.779779 10719 catalog_manager.cc:4841] TS d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): tablet e5757bc9ac9441c3bdfa6c5b2b743b6d (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:52:11.783751 10923 consensus_queue.cc:237] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:11.788846 11013 raft_consensus.cc:1261] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:52:11.789003 11635 raft_consensus.cc:1261] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 1 index: 4. (index mismatch)
I20260809 15:52:11.791373 12731 consensus_queue.cc:1035] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:52:11.792243 12878 consensus_queue.cc:1035] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:52:11.802316 12872 raft_consensus.cc:2932] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:11.807337 11013 raft_consensus.cc:2932] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:11.808703 11637 raft_consensus.cc:2932] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.4: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:11.815263 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 974451500a6e434f8d407d9f178ab710 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:52:11.830602 10737 catalog_manager.cc:5495] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 3 to 4, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:11.851871 10828 tablet_service.cc:1514] Processing DeleteTablet for tablet 974451500a6e434f8d407d9f178ab710 with delete_type TABLET_DATA_TOMBSTONED (TS a53441872c0c416b8e2f26f52399f536 not found in new config with opid_index 4) from {username='slave'} at 127.0.0.1:45572
I20260809 15:52:11.852924 12900 tablet_replica.cc:330] T 974451500a6e434f8d407d9f178ab710 P a53441872c0c416b8e2f26f52399f536: stopping tablet replica
I20260809 15:52:11.857396 10923 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:11.853983 12900 raft_consensus.cc:2229] T 974451500a6e434f8d407d9f178ab710 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 15:52:11.859102 12900 raft_consensus.cc:2258] T 974451500a6e434f8d407d9f178ab710 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 15:52:11.857996 10923 raft_consensus.cc:595] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:11.861974 10931 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:11.862521 10931 raft_consensus.cc:595] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:11.863673 12900 ts_tablet_manager.cc:1905] T 974451500a6e434f8d407d9f178ab710 P a53441872c0c416b8e2f26f52399f536: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:52:11.866128 10837 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:52:11.866659 10837 raft_consensus.cc:595] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:11.869704 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260809 15:52:11.871336 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:52:11.871955 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:11.878021 12900 ts_tablet_manager.cc:1918] T 974451500a6e434f8d407d9f178ab710 P a53441872c0c416b8e2f26f52399f536: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 1.3
I20260809 15:52:11.878479 12900 log.cc:1196] T 974451500a6e434f8d407d9f178ab710 P a53441872c0c416b8e2f26f52399f536: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-0-root/wals/974451500a6e434f8d407d9f178ab710
I20260809 15:52:11.880337 10722 catalog_manager.cc:4841] TS a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): tablet 974451500a6e434f8d407d9f178ab710 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:52:11.941027 12872 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:11.941560 12872 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:11.943598 12872 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:11.944525 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:11.944597 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:11.946131 10891 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:11.946767 12872 raft_consensus.cc:2726] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:11.995040 12731 raft_consensus.cc:979] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:11.995589 12872 raft_consensus.cc:1067] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:11.996976 11013 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50542
I20260809 15:52:11.997455 11013 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:11.997812 11013 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:12.005069 11013 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:12.007316 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:12.007800 10837 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:12.007958 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:12.008523 10927 raft_consensus.cc:3032] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:12.008783 10927 raft_consensus.cc:729] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:12.009454 10927 consensus_queue.cc:260] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:12.010476 10927 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:12.015859 10837 raft_consensus.cc:2454] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:12.017395 10978 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:12.018186 12770 raft_consensus.cc:2781] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:12.018667 12770 raft_consensus.cc:686] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:12.018708 10927 raft_consensus.cc:2454] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:12.019582 12770 consensus_queue.cc:237] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:12.006898 11013 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:12.035549 10737 catalog_manager.cc:5495] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 2 to 3, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 3 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:52:12.094429 10843 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:52:12.095593 12731 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:12.184300 11637 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:52:12.185505 12731 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260809 15:52:12.190244 10956 debug-util.cc:398] Leaking SignalData structure 0x7b080030ca80 after lost signal to thread 9556
W20260809 15:52:12.191149 10956 debug-util.cc:398] Leaking SignalData structure 0x7b0800342fe0 after lost signal to thread 10773
W20260809 15:52:12.192153 10956 debug-util.cc:398] Leaking SignalData structure 0x7b0800329320 after lost signal to thread 10875
W20260809 15:52:12.193202 10956 debug-util.cc:398] Leaking SignalData structure 0x7b08003378e0 after lost signal to thread 10959
W20260809 15:52:12.194918 10956 debug-util.cc:398] Leaking SignalData structure 0x7b080032c980 after lost signal to thread 11047
W20260809 15:52:12.195956 10956 debug-util.cc:398] Leaking SignalData structure 0x7b0800269a20 after lost signal to thread 11667
I20260809 15:52:12.209478 11013 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:52:12.212474 12879 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
W20260809 15:52:12.273102 10722 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: leader transfer in progress [suppressed 3 similar messages]
I20260809 15:52:12.278656 12826 raft_consensus.cc:979] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:12.279153 12826 raft_consensus.cc:1067] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:12.280889 11015 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50542
I20260809 15:52:12.281420 11015 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:12.281694 11015 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:12.288311 11015 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:12.290504 11015 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:12.291071 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:12.295583 10843 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:12.301071 10843 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:12.292788 12856 raft_consensus.cc:979] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:12.293648 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:12.302543 12855 raft_consensus.cc:1067] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:12.302834 11635 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:12.291710 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
W20260809 15:52:12.301302 10722 catalog_manager.cc:4584] TS 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): ChangeConfig:REMOVE_PEER RPC failed for tablet 0b2b0df042e44725a7c91deae3df1956: Remote error: Service unavailable: leader transfer in progress
I20260809 15:52:12.304526 10923 raft_consensus.cc:3032] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:12.304819 10923 raft_consensus.cc:729] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:12.305495 10923 consensus_queue.cc:260] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:12.307607 10923 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:12.308616 11635 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:12.309577 10978 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:12.311046 12858 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:12.311444 12858 raft_consensus.cc:686] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:12.312326 12858 consensus_queue.cc:237] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:12.322337 11015 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50528
I20260809 15:52:12.322921 11015 raft_consensus.cc:482] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:12.323249 11015 raft_consensus.cc:3037] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:12.325534 10736 catalog_manager.cc:5495] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 3 to 4, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 4 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:12.332382 10923 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:12.334364 11015 raft_consensus.cc:504] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:12.336098 11015 leader_election.cc:288] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:12.337109 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:12.337139 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:12.337823 11635 raft_consensus.cc:3037] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:12.337874 10837 raft_consensus.cc:3032] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:12.338390 10837 raft_consensus.cc:729] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:12.339174 10837 consensus_queue.cc:260] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:12.340420 10837 raft_consensus.cc:3037] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:12.346017 11635 raft_consensus.cc:2454] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:12.348825 10978 leader_election.cc:302] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:12.349656 12858 raft_consensus.cc:2781] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:12.350178 12858 raft_consensus.cc:686] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:12.351020 12858 consensus_queue.cc:237] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:12.353729 10719 catalog_manager.cc:5025] ChangeConfig:REMOVE_PEER RPC for tablet 0b2b0df042e44725a7c91deae3df1956 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260809 15:52:12.356249 11015 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:50526
I20260809 15:52:12.356784 11015 raft_consensus.cc:595] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 LEADER]: Received request to transfer leadership
I20260809 15:52:12.369516 11010 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:50526
I20260809 15:52:12.370076 11010 raft_consensus.cc:595] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 LEADER]: Received request to transfer leadership
I20260809 15:52:12.370528 10837 raft_consensus.cc:2454] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:12.381592 11015 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:50526
I20260809 15:52:12.382262 11015 raft_consensus.cc:595] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 LEADER]: Received request to transfer leadership
W20260809 15:52:12.383922 10789 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: leadership transfer for 0b2b0df042e44725a7c91deae3df1956 already in progress
W20260809 15:52:12.384259 10789 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 0b2b0df042e44725a7c91deae3df1956 already in progress
I20260809 15:52:12.385665 10736 catalog_manager.cc:5495] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 2 to 3, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 3 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:12.536849 10837 raft_consensus.cc:1261] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:52:12.538066 12678 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:12.552981 10931 raft_consensus.cc:1261] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:52:12.555956 12858 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:12.755543 10837 raft_consensus.cc:1261] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:12.756676 12678 consensus_queue.cc:1035] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:12.769461 11635 raft_consensus.cc:1261] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:12.774053 12678 consensus_queue.cc:1035] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:12.844141 10843 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:12.845403 12858 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:12.850255 12678 raft_consensus.cc:979] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:12.850736 12858 raft_consensus.cc:1067] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:12.852744 10843 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45578
I20260809 15:52:12.853348 10843 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:12.853659 10843 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:12.858819 11635 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:12.860536 12858 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:12.875738 10931 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:12.880211 12858 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:12.880688 10843 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:12.885524 10837 raft_consensus.cc:1226] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:12.890703 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:12.891238 10923 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:12.891894 12678 consensus_queue.cc:1046] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:12.892978 12926 raft_consensus.cc:3032] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:12.893333 12926 raft_consensus.cc:729] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:12.894282 12926 consensus_queue.cc:260] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:12.898650 10923 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:12.900394 10843 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:12.901924 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:12.952145 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:12.952947 11635 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:12.953390 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:12.961599 12926 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:13.027887 11010 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:13.031719 11635 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:13.058491 10811 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:13.060114 12932 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:13.060999 12932 raft_consensus.cc:686] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:13.075546 11015 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:52:13.076248 11015 raft_consensus.cc:595] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:52:13.068115 12932 consensus_queue.cc:237] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:13.086282 11015 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:52:13.086831 11015 raft_consensus.cc:595] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:13.088819 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260809 15:52:13.090422 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:52:13.090958 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:13.111584 10734 catalog_manager.cc:5495] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 4 to 5, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 5 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:13.463609 10927 consensus_queue.cc:237] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:13.470961 11015 raft_consensus.cc:1261] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 15:52:13.471372 10893 consensus_peers.cc:487] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cfa61e868cfb4915b0572f8e33745f43. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:52:13.473865 12731 consensus_queue.cc:1035] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:52:13.477181 10842 raft_consensus.cc:1261] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:52:13.478778 12731 consensus_queue.cc:1035] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:13.494145 12879 raft_consensus.cc:2932] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:13.501279 10842 raft_consensus.cc:2932] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:13.503051 12926 raft_consensus.cc:979] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:13.503567 12926 raft_consensus.cc:1067] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:13.504942 10837 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45578
I20260809 15:52:13.505470 10837 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:13.505779 10837 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:13.505069 11010 raft_consensus.cc:2932] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:13.529145 10737 catalog_manager.cc:5495] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:13.549873 10837 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:13.557677 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:13.558276 10931 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:13.558763 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:13.559327 11010 raft_consensus.cc:3032] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:13.559605 11010 raft_consensus.cc:729] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:13.560308 11010 consensus_queue.cc:260] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:13.561305 11010 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:13.567283 10931 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:13.571132 10811 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:13.557112 10837 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:13.572665 12934 raft_consensus.cc:2781] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:13.573124 12934 raft_consensus.cc:686] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
W20260809 15:52:13.573737 10722 catalog_manager.cc:5025] ChangeConfig:REMOVE_PEER RPC for tablet 0b2b0df042e44725a7c91deae3df1956 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260809 15:52:13.569512 11015 consensus_queue.cc:237] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:13.573953 12934 consensus_queue.cc:237] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:13.579063 11010 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
W20260809 15:52:13.595425 10978 consensus_peers.cc:487] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 08f368b4f4e748c596049c81eaed3f70. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:52:13.596976 10837 raft_consensus.cc:1261] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260809 15:52:13.599298 12678 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:52:13.607818 10931 raft_consensus.cc:1261] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260809 15:52:13.613096 11010 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260809 15:52:13.615365 12678 raft_consensus.cc:979] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:13.615746 12678 raft_consensus.cc:1067] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:13.616864 12932 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 15:52:13.624059 12926 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:52:13.634968 10931 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54236
I20260809 15:52:13.635512 10931 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:13.635856 10931 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:13.649416 12926 raft_consensus.cc:2932] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:13.653793 10734 catalog_manager.cc:5495] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 3 to 4, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 4 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:13.656541 10837 raft_consensus.cc:2932] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:13.665022 10927 raft_consensus.cc:2932] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:13.676512 11642 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260809 15:52:13.679991 12934 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.002s
I20260809 15:52:13.692396 11015 consensus_queue.cc:237] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:13.707062 10734 catalog_manager.cc:5495] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index -1 to 4, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 3 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:13.662688 10931 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:13.723811 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:13.724435 11014 raft_consensus.cc:3032] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:13.724727 11014 raft_consensus.cc:729] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:13.722996 10927 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260809 15:52:13.725445 11014 consensus_queue.cc:260] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:13.726879 11014 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:13.730554 12933 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 15:52:13.737186 11014 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:13.747087 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:13.749199 11636 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:13.756543 11636 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:13.745998 10931 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:13.757718 10893 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:13.758705 12944 raft_consensus.cc:2781] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:52:13.759158 12944 raft_consensus.cc:686] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:13.759976 12944 consensus_queue.cc:237] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:13.848032 10736 catalog_manager.cc:5495] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 1 to 2, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:13.854807 10923 raft_consensus.cc:1261] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260809 15:52:13.855223 10843 raft_consensus.cc:1261] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
W20260809 15:52:13.856724 10978 consensus_peers.cc:487] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5b472f5780ac4f56a2443e8ccc49125f. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:52:13.857506 12928 consensus_queue.cc:1035] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:13.858646 12926 consensus_queue.cc:1035] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:13.884670 12949 raft_consensus.cc:2932] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:13.908193 10837 consensus_queue.cc:237] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:13.893258 10923 raft_consensus.cc:2932] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:13.926321 11015 raft_consensus.cc:1261] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260809 15:52:13.929883 10931 raft_consensus.cc:1261] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260809 15:52:13.931407 12934 consensus_queue.cc:1035] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:13.933421 12934 consensus_queue.cc:1035] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:13.944435 10843 raft_consensus.cc:2932] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
W20260809 15:52:13.949209 10811 consensus_peers.cc:487] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 908409e5747b470cabbc8aed443397f7. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:52:13.963439 12932 raft_consensus.cc:2932] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:13.964805 10736 catalog_manager.cc:5495] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 2 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:13.971800 10923 raft_consensus.cc:2932] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:14.026504 11014 raft_consensus.cc:1261] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:52:14.029088 12933 consensus_queue.cc:1035] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:14.050230 11015 raft_consensus.cc:2932] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Committing config change with OpId 2.3: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:14.068256 10734 catalog_manager.cc:5495] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index -1 to 3, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:14.076964 12962 ts_tablet_manager.cc:927] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:14.095577 10927 raft_consensus.cc:1261] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:52:14.112051 12934 consensus_queue.cc:1035] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:52:14.118656 12965 ts_tablet_manager.cc:927] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:14.141700 11636 raft_consensus.cc:1261] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:14.143283 12879 consensus_queue.cc:1035] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:14.140630 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:14.145123 12962 tablet_copy_client.cc:316] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.3:39223
I20260809 15:52:14.151119 11027 tablet_copy_service.cc:140] P d74944fa9fa14ec78b0640ea709ff3aa: Received BeginTabletCopySession request for tablet 08f368b4f4e748c596049c81eaed3f70 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:39978)
I20260809 15:52:14.152019 11027 tablet_copy_service.cc:161] P d74944fa9fa14ec78b0640ea709ff3aa: Beginning new tablet copy session on tablet 08f368b4f4e748c596049c81eaed3f70 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:39978: session id = c7aa762318cb4e97875333f10679a9f9-08f368b4f4e748c596049c81eaed3f70
I20260809 15:52:14.178823 11027 tablet_copy_source_session.cc:205] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:52:14.179107 11014 raft_consensus.cc:1261] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:14.179796 12965 tablet_copy_client.cc:316] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.2:32783
I20260809 15:52:14.180760 12944 consensus_queue.cc:1035] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:14.181682 10942 tablet_copy_service.cc:140] P 20730d75ad954a82b126148f8152a9f3: Received BeginTabletCopySession request for tablet cfa61e868cfb4915b0572f8e33745f43 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:52758)
I20260809 15:52:14.182519 10942 tablet_copy_service.cc:161] P 20730d75ad954a82b126148f8152a9f3: Beginning new tablet copy session on tablet cfa61e868cfb4915b0572f8e33745f43 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:52758: session id = c7aa762318cb4e97875333f10679a9f9-cfa61e868cfb4915b0572f8e33745f43
I20260809 15:52:14.185724 12962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 08f368b4f4e748c596049c81eaed3f70. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:52:14.201041 10942 tablet_copy_source_session.cc:205] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:52:14.217103 12965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cfa61e868cfb4915b0572f8e33745f43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:52:14.221742 12962 tablet_copy_client.cc:791] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:52:14.222431 12962 tablet_copy_client.cc:655] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:52:14.242369 12962 tablet_copy_client.cc:531] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:52:14.289230 12962 tablet_bootstrap.cc:492] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:52:14.290704 12965 tablet_copy_client.cc:791] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:52:14.291725 12965 tablet_copy_client.cc:655] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:52:14.296797 12965 tablet_copy_client.cc:531] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:52:14.327639 12965 tablet_bootstrap.cc:492] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:52:14.356084 10843 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:52:14.356910 10843 raft_consensus.cc:595] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:14.364349 10843 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:52:14.365049 10843 raft_consensus.cc:595] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:14.375192 10927 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:14.375882 10927 raft_consensus.cc:595] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:14.377140 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 3
I20260809 15:52:14.379015 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:52:14.379834 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:14.402376 12933 raft_consensus.cc:979] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:14.403039 12933 raft_consensus.cc:1067] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:14.403702 12962 tablet_bootstrap.cc:492] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:52:14.404798 12962 tablet_bootstrap.cc:492] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:52:14.405469 11014 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50528
I20260809 15:52:14.406031 11014 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:14.406428 11014 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:14.414469 11014 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:14.419144 12962 ts_tablet_manager.cc:1397] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.130s user 0.023s sys 0.021s
I20260809 15:52:14.426447 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:14.428282 10843 raft_consensus.cc:3032] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:14.428602 10843 raft_consensus.cc:729] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:14.429442 10843 consensus_queue.cc:260] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:14.431321 10843 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:14.435729 12974 ts_tablet_manager.cc:927] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:14.446774 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:14.447510 11636 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:14.450197 11014 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:14.450709 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:14.451273 10923 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:14.453212 10843 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:14.457479 12974 tablet_copy_client.cc:316] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.3:39223
I20260809 15:52:14.461314 11027 tablet_copy_service.cc:140] P d74944fa9fa14ec78b0640ea709ff3aa: Received BeginTabletCopySession request for tablet 5b472f5780ac4f56a2443e8ccc49125f from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:39978)
I20260809 15:52:14.448787 12962 raft_consensus.cc:348] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:14.462006 12962 raft_consensus.cc:729] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:52:14.462028 11027 tablet_copy_service.cc:161] P d74944fa9fa14ec78b0640ea709ff3aa: Beginning new tablet copy session on tablet 5b472f5780ac4f56a2443e8ccc49125f from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:39978: session id = c7aa762318cb4e97875333f10679a9f9-5b472f5780ac4f56a2443e8ccc49125f
I20260809 15:52:14.466668 12962 consensus_queue.cc:260] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:14.476994 11027 tablet_copy_source_session.cc:205] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:52:14.465548 10923 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:14.479148 10978 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:14.464274 11636 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:14.481376 12678 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:14.481796 12678 raft_consensus.cc:686] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:14.482648 12678 consensus_queue.cc:237] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:14.489737 12962 ts_tablet_manager.cc:1428] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.051s user 0.004s sys 0.006s
I20260809 15:52:14.494750 12965 tablet_bootstrap.cc:492] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:52:14.496310 11028 tablet_copy_service.cc:342] P d74944fa9fa14ec78b0640ea709ff3aa: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-08f368b4f4e748c596049c81eaed3f70 received from {username='slave'} at 127.0.0.1:39978
I20260809 15:52:14.503513 11028 tablet_copy_service.cc:433] P d74944fa9fa14ec78b0640ea709ff3aa: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-08f368b4f4e748c596049c81eaed3f70 on tablet 08f368b4f4e748c596049c81eaed3f70 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:14.503556 12965 tablet_bootstrap.cc:492] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:52:14.504781 12965 ts_tablet_manager.cc:1397] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.177s user 0.026s sys 0.037s
I20260809 15:52:14.498009 10736 catalog_manager.cc:5495] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 5 to 6, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 6 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:14.532891 12974 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b472f5780ac4f56a2443e8ccc49125f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:52:14.542309 12965 raft_consensus.cc:348] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:14.543300 12965 raft_consensus.cc:729] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:52:14.543995 12965 consensus_queue.cc:260] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:14.546533 12965 ts_tablet_manager.cc:1428] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.041s user 0.010s sys 0.000s
I20260809 15:52:14.559248 11635 raft_consensus.cc:1203] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4] Dedup: 3.4->[]
I20260809 15:52:14.573895 10942 tablet_copy_service.cc:342] P 20730d75ad954a82b126148f8152a9f3: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-cfa61e868cfb4915b0572f8e33745f43 received from {username='slave'} at 127.0.0.1:52758
I20260809 15:52:14.574386 10942 tablet_copy_service.cc:433] P 20730d75ad954a82b126148f8152a9f3: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-cfa61e868cfb4915b0572f8e33745f43 on tablet cfa61e868cfb4915b0572f8e33745f43 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:14.567660 12962 ts_tablet_manager.cc:927] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615)
I20260809 15:52:14.604230 12962 tablet_copy_client.cc:316] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.1:46615
I20260809 15:52:14.605981 10858 tablet_copy_service.cc:140] P a53441872c0c416b8e2f26f52399f536: Received BeginTabletCopySession request for tablet 908409e5747b470cabbc8aed443397f7 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:54342)
I20260809 15:52:14.606566 10858 tablet_copy_service.cc:161] P a53441872c0c416b8e2f26f52399f536: Beginning new tablet copy session on tablet 908409e5747b470cabbc8aed443397f7 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:54342: session id = c7aa762318cb4e97875333f10679a9f9-908409e5747b470cabbc8aed443397f7
I20260809 15:52:14.616683 10858 tablet_copy_source_session.cc:205] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:52:14.619997 12962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 908409e5747b470cabbc8aed443397f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:52:14.620194 12974 tablet_copy_client.cc:791] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:52:14.620807 12974 tablet_copy_client.cc:655] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:52:14.632683 12974 tablet_copy_client.cc:531] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
W20260809 15:52:14.637979 10719 catalog_manager.cc:5025] ChangeConfig:REMOVE_PEER RPC for tablet 0b2b0df042e44725a7c91deae3df1956 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223) due to error UNKNOWN_ERROR; will retry: Illegal state: Leader has not yet committed an operation in its own term
I20260809 15:52:14.646812 12974 tablet_bootstrap.cc:492] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:52:14.682686 12962 tablet_copy_client.cc:791] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:52:14.683502 12962 tablet_copy_client.cc:655] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:52:14.687844 12962 tablet_copy_client.cc:531] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:52:14.690434 12974 tablet_bootstrap.cc:492] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:52:14.695580 12974 tablet_bootstrap.cc:492] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:52:14.696240 12974 ts_tablet_manager.cc:1397] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.050s user 0.033s sys 0.004s
I20260809 15:52:14.698925 12974 raft_consensus.cc:348] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:14.699959 12974 raft_consensus.cc:729] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:52:14.700600 12974 consensus_queue.cc:260] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:14.703158 12974 ts_tablet_manager.cc:1428] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260809 15:52:14.710788 12962 tablet_bootstrap.cc:492] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:52:14.711046 11027 tablet_copy_service.cc:342] P d74944fa9fa14ec78b0640ea709ff3aa: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-5b472f5780ac4f56a2443e8ccc49125f received from {username='slave'} at 127.0.0.1:39978
I20260809 15:52:14.711488 11027 tablet_copy_service.cc:433] P d74944fa9fa14ec78b0640ea709ff3aa: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-5b472f5780ac4f56a2443e8ccc49125f on tablet 5b472f5780ac4f56a2443e8ccc49125f with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:14.746819 12962 tablet_bootstrap.cc:492] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:52:14.750066 12962 tablet_bootstrap.cc:492] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:52:14.750869 12962 ts_tablet_manager.cc:1397] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.040s user 0.017s sys 0.012s
I20260809 15:52:14.752087 12872 raft_consensus.cc:979] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:14.752636 12944 raft_consensus.cc:1067] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:14.753870 12962 raft_consensus.cc:348] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:14.754869 11010 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50542
I20260809 15:52:14.754901 12962 raft_consensus.cc:729] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:52:14.755425 11010 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:14.755797 11010 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:14.755929 12962 consensus_queue.cc:260] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:14.764046 12962 ts_tablet_manager.cc:1428] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.013s user 0.005s sys 0.000s
I20260809 15:52:14.766047 10858 tablet_copy_service.cc:342] P a53441872c0c416b8e2f26f52399f536: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-908409e5747b470cabbc8aed443397f7 received from {username='slave'} at 127.0.0.1:54342
I20260809 15:52:14.766448 10858 tablet_copy_service.cc:433] P a53441872c0c416b8e2f26f52399f536: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-908409e5747b470cabbc8aed443397f7 on tablet 908409e5747b470cabbc8aed443397f7 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:14.769744 11010 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:14.779667 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:14.780702 10927 raft_consensus.cc:3032] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:14.781044 10927 raft_consensus.cc:729] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:14.781821 10927 consensus_queue.cc:260] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:14.782418 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:14.783910 10927 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:14.783998 11010 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:14.784560 10842 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:14.796892 10842 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:14.797909 10978 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:14.798652 12979 raft_consensus.cc:2781] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:52:14.799182 12979 raft_consensus.cc:686] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:14.799958 12979 consensus_queue.cc:237] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:14.801245 10927 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:14.821060 10736 catalog_manager.cc:5495] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 1 to 2, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 2 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:52:14.874624 12951 raft_consensus.cc:979] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:14.875144 12934 raft_consensus.cc:1067] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:14.877089 10923 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54224
I20260809 15:52:14.877676 10923 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:14.878023 10923 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:14.892908 10923 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:14.895311 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:14.895923 10843 raft_consensus.cc:3032] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:14.895884 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:14.896503 10843 raft_consensus.cc:729] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:14.896763 11635 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:14.897212 10923 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:14.897325 10843 consensus_queue.cc:260] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:14.898442 10843 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:14.903273 10843 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:14.904340 10893 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:14.905133 12944 raft_consensus.cc:2781] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:14.905589 12944 raft_consensus.cc:686] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:14.906492 12944 consensus_queue.cc:237] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:14.921116 11635 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:14.922472 10736 catalog_manager.cc:5495] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 2 to 3, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 3 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:14.979303 10843 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260809 15:52:14.981070 12926 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 15:52:15.015352 12926 raft_consensus.cc:1050] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:52:15.017652 12926 consensus_queue.cc:237] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:15.029080 11636 raft_consensus.cc:1261] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 3 LEARNER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:52:15.029453 11635 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260809 15:52:15.031947 12979 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:15.032027 12678 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260809 15:52:15.041229 10843 raft_consensus.cc:1261] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:52:15.045557 11642 raft_consensus.cc:1203] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20260809 15:52:15.050638 10923 raft_consensus.cc:1261] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:52:15.052152 12926 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:15.056885 12678 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:15.095989 11636 raft_consensus.cc:1203] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:52:15.110479 10930 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260809 15:52:15.112818 12979 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 15:52:15.123361 11642 raft_consensus.cc:1203] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Deduplicated request from leader. Original: 2.2->[2.3-2.3] Dedup: 2.3->[]
I20260809 15:52:15.209358 12678 raft_consensus.cc:2932] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:15.211916 10923 raft_consensus.cc:2932] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:15.229415 11635 raft_consensus.cc:2932] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:15.229880 10737 catalog_manager.cc:5495] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 4 to 5, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 3 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:15.259483 10842 raft_consensus.cc:2932] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:15.271008 11010 consensus_queue.cc:237] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:15.276835 11635 raft_consensus.cc:1261] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:15.278327 12926 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:15.283409 10843 raft_consensus.cc:1261] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:15.284801 12926 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:15.287889 12678 raft_consensus.cc:2932] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:15.297968 10923 raft_consensus.cc:1261] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:52:15.299402 12979 consensus_queue.cc:1035] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:15.303399 10719 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 08f368b4f4e748c596049c81eaed3f70 with cas_config_opid_index 5: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:52:15.305119 11636 raft_consensus.cc:2932] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:15.311056 10734 catalog_manager.cc:5495] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index 5 to 6, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New cstate: current_term: 3 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:15.313452 10843 raft_consensus.cc:2932] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Committing config change with OpId 3.6: config changed from index 5 to 6, VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) evicted. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:15.327144 10913 tablet_service.cc:1514] Processing DeleteTablet for tablet 08f368b4f4e748c596049c81eaed3f70 with delete_type TABLET_DATA_TOMBSTONED (TS 20730d75ad954a82b126148f8152a9f3 not found in new config with opid_index 6) from {username='slave'} at 127.0.0.1:54216
I20260809 15:52:15.329084 10837 raft_consensus.cc:1261] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:52:15.334111 10848 raft_consensus.cc:1261] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:15.342344 12944 consensus_queue.cc:1035] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:15.343544 12979 consensus_queue.cc:1035] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
W20260809 15:52:15.405200 13000 log.cc:924] Time spent T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.065s user 0.000s sys 0.000s
I20260809 15:52:15.413584 13001 tablet_replica.cc:330] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3: stopping tablet replica
I20260809 15:52:15.414726 13001 raft_consensus.cc:2229] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Raft consensus shutting down.
I20260809 15:52:15.415361 13001 raft_consensus.cc:2258] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Raft consensus is shut down!
I20260809 15:52:15.426280 13001 ts_tablet_manager.cc:1905] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:52:15.444178 11635 raft_consensus.cc:1261] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:15.459077 12879 consensus_queue.cc:1035] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260809 15:52:15.503479 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 66331.2us
I20260809 15:52:15.511795 12926 raft_consensus.cc:1050] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:52:15.511974 12933 raft_consensus.cc:1050] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:52:15.515595 12933 consensus_queue.cc:237] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:15.517024 12926 consensus_queue.cc:237] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 2.3, Last appended by leader: 1, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:15.552733 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
W20260809 15:52:15.577205 13006 log.cc:924] Time spent T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.099s user 0.001s sys 0.000s
W20260809 15:52:15.583602 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 47716.4us
I20260809 15:52:15.642719 13001 ts_tablet_manager.cc:1918] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 3.5
I20260809 15:52:15.643340 13001 log.cc:1196] T 08f368b4f4e748c596049c81eaed3f70 P 20730d75ad954a82b126148f8152a9f3: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-1-root/wals/08f368b4f4e748c596049c81eaed3f70
I20260809 15:52:15.644265 12944 raft_consensus.cc:1050] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:52:15.645526 10722 catalog_manager.cc:4841] TS 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): tablet 08f368b4f4e748c596049c81eaed3f70 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:52:15.646981 12944 consensus_queue.cc:237] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:15.670307 11635 raft_consensus.cc:1261] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:52:15.693652 13016 consensus_queue.cc:1035] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:15.705811 10923 raft_consensus.cc:1261] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:52:15.708945 11635 raft_consensus.cc:1261] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 2 LEARNER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:52:15.713510 10923 raft_consensus.cc:1261] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:52:15.713426 11010 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:52:15.715337 11010 raft_consensus.cc:595] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:15.714532 13010 consensus_queue.cc:1035] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.003s
I20260809 15:52:15.720512 13016 consensus_queue.cc:1035] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:15.722054 10931 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:15.722692 11010 raft_consensus.cc:1261] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:52:15.723229 12926 consensus_queue.cc:1035] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:15.726248 10931 raft_consensus.cc:595] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:52:15.733037 11015 raft_consensus.cc:1261] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:15.733095 10842 raft_consensus.cc:1261] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:52:15.734612 12872 consensus_queue.cc:1035] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:52:15.749692 13009 consensus_queue.cc:1035] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:15.756862 10848 raft_consensus.cc:1261] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:15.757359 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260809 15:52:15.756829 11636 raft_consensus.cc:1261] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:15.759238 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:52:15.759821 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:15.761746 13016 consensus_queue.cc:1035] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:15.771211 13014 consensus_queue.cc:1035] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:52:15.812513 13014 consensus_queue.cc:1035] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:15.819248 10927 raft_consensus.cc:2932] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:15.812628 12933 raft_consensus.cc:2932] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:15.933389 10847 raft_consensus.cc:2932] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:15.941244 11635 raft_consensus.cc:2932] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:15.942770 13009 raft_consensus.cc:2932] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:15.949960 11015 raft_consensus.cc:2932] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:15.950042 10733 catalog_manager.cc:5495] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:15.982255 13017 raft_consensus.cc:979] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:15.983302 12944 raft_consensus.cc:1067] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:15.985311 12879 raft_consensus.cc:2932] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:16.003182 10930 raft_consensus.cc:2932] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:16.014773 11635 raft_consensus.cc:2932] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Committing config change with OpId 2.4: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:16.021677 10847 consensus_queue.cc:237] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:16.031747 10931 raft_consensus.cc:1261] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:16.036255 10843 raft_consensus.cc:2932] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:16.037431 11015 raft_consensus.cc:2932] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:16.038727 11637 raft_consensus.cc:1261] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:16.038998 12933 consensus_queue.cc:1035] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:16.041783 10847 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45594
I20260809 15:52:16.049259 13022 consensus_queue.cc:1035] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:16.050122 10847 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:16.050534 10847 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:16.057211 10847 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:16.062510 10737 catalog_manager.cc:5495] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index 3 to 4, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 2 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:16.070430 11637 raft_consensus.cc:2932] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:16.084527 11010 consensus_queue.cc:237] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:16.092252 11642 raft_consensus.cc:1261] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:16.094152 13007 consensus_queue.cc:1035] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:16.095764 10923 raft_consensus.cc:1261] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:16.103575 10737 catalog_manager.cc:5495] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:16.103406 13007 raft_consensus.cc:2932] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:16.104462 12933 raft_consensus.cc:2932] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:16.115893 10930 raft_consensus.cc:2932] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:16.127815 12926 raft_consensus.cc:979] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:16.128268 12926 raft_consensus.cc:1067] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:16.121693 11642 raft_consensus.cc:2932] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:16.148119 13010 consensus_queue.cc:1035] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.004s
I20260809 15:52:16.160928 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:16.161597 10927 raft_consensus.cc:3032] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:16.161981 10927 raft_consensus.cc:729] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:16.154140 10923 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54236
I20260809 15:52:16.163201 10923 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:16.162788 10927 consensus_queue.cc:260] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:16.163543 10923 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:16.169900 10923 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:16.125722 11635 raft_consensus.cc:2932] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:16.172950 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:16.173502 11010 raft_consensus.cc:3032] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:16.173787 11010 raft_consensus.cc:729] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:16.174626 11010 consensus_queue.cc:260] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:16.175657 11010 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:16.189648 11010 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:16.190712 10847 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:16.191375 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:16.223096 11014 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:16.194860 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 908409e5747b470cabbc8aed443397f7 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260809 15:52:16.197715 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 45182.8us
I20260809 15:52:16.230818 11014 raft_consensus.cc:2454] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:16.200956 10719 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 5b472f5780ac4f56a2443e8ccc49125f with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
I20260809 15:52:16.232965 10891 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:16.233762 12879 raft_consensus.cc:2781] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:52:16.234310 12879 raft_consensus.cc:686] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:16.235201 12879 consensus_queue.cc:237] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:16.237143 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:16.237743 10847 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:16.201239 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:16.238314 11636 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:16.246099 10930 raft_consensus.cc:2932] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 4 to 5, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:16.258754 11636 raft_consensus.cc:2454] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:16.210920 10736 catalog_manager.cc:5495] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index 4 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:16.216847 10927 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:16.219231 10737 catalog_manager.cc:5495] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index 4 to 5, VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1) evicted. New cstate: current_term: 2 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:16.198148 10923 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:16.275983 10828 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b472f5780ac4f56a2443e8ccc49125f with delete_type TABLET_DATA_TOMBSTONED (TS a53441872c0c416b8e2f26f52399f536 not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:45572
I20260809 15:52:16.277673 10847 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:16.279745 10927 raft_consensus.cc:2454] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:16.280647 10809 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:16.284497 12934 raft_consensus.cc:2781] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:52:16.285003 12934 raft_consensus.cc:686] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:16.285959 12934 consensus_queue.cc:237] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 2, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:16.288910 10722 catalog_manager.cc:5025] ChangeConfig:REMOVE_PEER RPC for tablet cfa61e868cfb4915b0572f8e33745f43 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER failed with leader 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica 20730d75ad954a82b126148f8152a9f3 is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 2 committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:16.299391 10998 tablet_service.cc:1514] Processing DeleteTablet for tablet 908409e5747b470cabbc8aed443397f7 with delete_type TABLET_DATA_TOMBSTONED (TS d74944fa9fa14ec78b0640ea709ff3aa not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:50526
I20260809 15:52:16.309721 13041 tablet_replica.cc:330] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536: stopping tablet replica
I20260809 15:52:16.312206 13041 raft_consensus.cc:2229] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 15:52:16.311312 10734 catalog_manager.cc:5495] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 1 to 2, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 2 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:16.317121 13041 raft_consensus.cc:2258] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:52:16.316906 10736 catalog_manager.cc:5495] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 1 to 2, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:16.319456 13042 tablet_replica.cc:330] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa: stopping tablet replica
I20260809 15:52:16.320502 13042 raft_consensus.cc:2229] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 15:52:16.322049 13041 ts_tablet_manager.cc:1905] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:52:16.326390 13042 raft_consensus.cc:2258] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:52:16.339607 13042 ts_tablet_manager.cc:1905] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:52:16.347735 13041 ts_tablet_manager.cc:1918] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260809 15:52:16.348244 13041 log.cc:1196] T 5b472f5780ac4f56a2443e8ccc49125f P a53441872c0c416b8e2f26f52399f536: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-0-root/wals/5b472f5780ac4f56a2443e8ccc49125f
I20260809 15:52:16.349957 10722 catalog_manager.cc:4841] TS a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615): tablet 5b472f5780ac4f56a2443e8ccc49125f (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:52:16.370497 13042 ts_tablet_manager.cc:1918] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260809 15:52:16.371001 13042 log.cc:1196] T 908409e5747b470cabbc8aed443397f7 P d74944fa9fa14ec78b0640ea709ff3aa: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-2-root/wals/908409e5747b470cabbc8aed443397f7
I20260809 15:52:16.372992 10719 catalog_manager.cc:4841] TS d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): tablet 908409e5747b470cabbc8aed443397f7 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:52:16.632390 11014 raft_consensus.cc:1261] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:52:16.633934 13035 consensus_queue.cc:1035] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:16.635334 10847 raft_consensus.cc:1261] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260809 15:52:16.637120 13035 consensus_queue.cc:1035] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:16.703050 11636 raft_consensus.cc:1261] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:52:16.705657 12934 consensus_queue.cc:1035] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:52:16.721292 10927 raft_consensus.cc:1261] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:52:16.722831 13024 consensus_queue.cc:1035] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
W20260809 15:52:16.750244 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 47823.1us
I20260809 15:52:16.764894 11010 raft_consensus.cc:1261] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 2 index: 4. (index mismatch)
I20260809 15:52:16.778666 12933 consensus_queue.cc:1035] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:16.828509 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
W20260809 15:52:16.848848 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 71509.4us
I20260809 15:52:16.959599 10848 consensus_queue.cc:237] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 3, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:16.966316 11636 raft_consensus.cc:1261] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:16.967146 10930 raft_consensus.cc:1261] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:16.970258 13054 consensus_queue.cc:1035] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:16.971052 12934 consensus_queue.cc:1035] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:16.991734 12933 raft_consensus.cc:2932] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:16.997289 10930 raft_consensus.cc:2932] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:17.000748 11636 raft_consensus.cc:2932] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Committing config change with OpId 2.5: config changed from index 3 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:17.021380 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet cfa61e868cfb4915b0572f8e33745f43 with cas_config_opid_index 3: ChangeConfig:REMOVE_PEER succeeded (attempt 6)
I20260809 15:52:17.035193 10734 catalog_manager.cc:5495] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 3 to 5, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New cstate: current_term: 2 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:17.040853 10930 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:17.041474 10930 raft_consensus.cc:595] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:17.045275 10842 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:52:17.045972 10842 raft_consensus.cc:595] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:17.047992 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260809 15:52:17.050282 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:52:17.051332 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:17.054044 13050 raft_consensus.cc:979] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:17.054916 13050 raft_consensus.cc:1067] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:17.056663 11014 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50542
I20260809 15:52:17.057153 11014 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:17.057479 11014 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:17.071133 11014 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:17.072744 11014 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:17.074630 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:17.075232 10930 raft_consensus.cc:3032] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:17.075508 10930 raft_consensus.cc:729] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:17.076205 10930 consensus_queue.cc:260] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:17.077438 10930 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:17.078519 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:17.079116 10842 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:17.095485 10930 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:17.093240 10842 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:17.096683 10978 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:17.097573 12926 raft_consensus.cc:2781] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:17.101085 12926 raft_consensus.cc:686] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:17.102237 12926 consensus_queue.cc:237] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:17.122169 10998 tablet_service.cc:1514] Processing DeleteTablet for tablet cfa61e868cfb4915b0572f8e33745f43 with delete_type TABLET_DATA_TOMBSTONED (TS d74944fa9fa14ec78b0640ea709ff3aa not found in new config with opid_index 5) from {username='slave'} at 127.0.0.1:50526
I20260809 15:52:17.166597 13053 raft_consensus.cc:979] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:17.167469 13053 raft_consensus.cc:1067] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:17.178627 13058 tablet_replica.cc:330] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa: stopping tablet replica
I20260809 15:52:17.184470 13058 raft_consensus.cc:2229] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Raft consensus shutting down.
I20260809 15:52:17.190639 13058 raft_consensus.cc:2258] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Raft consensus is shut down!
I20260809 15:52:17.193737 10927 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54224
I20260809 15:52:17.194437 10927 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:17.194872 10927 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:17.177497 10737 catalog_manager.cc:5495] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 2 to 3, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 3 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:52:17.202252 10927 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:17.207628 13058 ts_tablet_manager.cc:1905] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:52:17.212275 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:17.212913 10848 raft_consensus.cc:3032] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:17.213199 10848 raft_consensus.cc:729] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:17.213967 10848 consensus_queue.cc:260] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:17.215206 10848 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
W20260809 15:52:17.220974 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 51196.1us
I20260809 15:52:17.223749 10927 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:17.225023 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:17.225651 11014 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:17.235848 11014 raft_consensus.cc:2454] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:17.238739 10891 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:17.239550 13050 raft_consensus.cc:2781] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:17.240022 13050 raft_consensus.cc:686] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:17.240859 13050 consensus_queue.cc:237] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:17.244127 10848 raft_consensus.cc:2454] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
W20260809 15:52:17.288537 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 54383.6us
I20260809 15:52:17.290989 13058 ts_tablet_manager.cc:1918] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 2.4
I20260809 15:52:17.293551 13058 log.cc:1196] T cfa61e868cfb4915b0572f8e33745f43 P d74944fa9fa14ec78b0640ea709ff3aa: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-2-root/wals/cfa61e868cfb4915b0572f8e33745f43
I20260809 15:52:17.298337 10719 catalog_manager.cc:4841] TS d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): tablet cfa61e868cfb4915b0572f8e33745f43 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:52:17.303484 10737 catalog_manager.cc:5495] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 3 to 4, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 4 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:17.493875 10848 raft_consensus.cc:1261] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:52:17.495808 13010 consensus_queue.cc:1035] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:17.547740 10930 raft_consensus.cc:1261] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:52:17.556648 13007 consensus_queue.cc:1035] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260809 15:52:17.655812 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 57578.5us
I20260809 15:52:17.774833 11013 raft_consensus.cc:1261] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:52:17.776647 13067 consensus_queue.cc:1035] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:17.851488 10848 raft_consensus.cc:1261] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:52:17.871935 13036 consensus_queue.cc:1035] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:52:18.252115 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:18.325451 10842 consensus_queue.cc:237] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:18.362592 11014 raft_consensus.cc:1261] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 15:52:18.363938 10811 consensus_peers.cc:487] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8975b3e7ddc544bc888dd61fffc5313b. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:52:18.364244 12934 consensus_queue.cc:1035] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:18.367465 10930 raft_consensus.cc:1261] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 15:52:18.369004 12933 consensus_queue.cc:1035] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s
I20260809 15:52:18.419934 11013 raft_consensus.cc:2932] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:18.418272 12933 raft_consensus.cc:2932] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:18.440062 10737 catalog_manager.cc:5495] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:18.446467 10930 raft_consensus.cc:2932] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.2: config changed from index -1 to 2, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:18.506525 10930 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:18.507333 10930 raft_consensus.cc:595] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:52:18.511216 10842 consensus_queue.cc:237] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:18.540772 11013 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "6508f17228904310be3c95818973f3a1"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
mode: GRACEFUL
new_leader_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:35856
I20260809 15:52:18.541343 11013 raft_consensus.cc:595] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Received request to transfer leadership to TS c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:18.557209 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260809 15:52:18.559196 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:52:18.559934 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
W20260809 15:52:18.572264 10811 consensus_peers.cc:487] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 5e8d12b5ef9e415cbcb25e62ab228546. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:52:18.624969 11013 raft_consensus.cc:1261] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260809 15:52:18.626559 13070 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:18.631242 10930 raft_consensus.cc:1261] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260809 15:52:18.634390 13050 raft_consensus.cc:979] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:18.634945 13050 raft_consensus.cc:1067] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:18.639078 13070 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:18.644515 10842 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45594
I20260809 15:52:18.645117 10842 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:18.647125 10842 raft_consensus.cc:3037] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:18.659874 10930 raft_consensus.cc:2932] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:18.665635 11013 raft_consensus.cc:2932] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:18.652561 13070 raft_consensus.cc:2932] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Committing config change with OpId 4.5: config changed from index -1 to 5, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:18.751075 10843 consensus_queue.cc:237] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:18.757728 10842 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:18.760200 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:18.760409 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:18.760216 10842 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:18.766151 10734 catalog_manager.cc:5495] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index -1 to 5, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 4 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:18.767835 11013 raft_consensus.cc:3037] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:18.760789 10930 raft_consensus.cc:3032] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:18.770147 10930 raft_consensus.cc:729] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:18.770898 10930 consensus_queue.cc:260] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:18.772149 10930 raft_consensus.cc:3037] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:18.780830 11013 raft_consensus.cc:2454] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:18.782581 10809 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:18.783936 13070 raft_consensus.cc:2781] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:18.784514 13070 raft_consensus.cc:686] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:18.785746 13070 consensus_queue.cc:237] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:18.804663 10930 raft_consensus.cc:2454] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:18.835053 10734 catalog_manager.cc:5495] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 2 to 3, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 3 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
W20260809 15:52:18.845405 10811 consensus_peers.cc:487] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 1f9b7d9e57934596b377ee8dc79ea23e. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:52:18.850268 11014 raft_consensus.cc:1261] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260809 15:52:18.852703 13070 consensus_queue.cc:1035] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:52:18.855257 10923 raft_consensus.cc:1261] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260809 15:52:18.861081 13070 consensus_queue.cc:1035] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:18.888708 13053 raft_consensus.cc:2932] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:18.896878 10923 raft_consensus.cc:2932] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:18.916188 10927 consensus_queue.cc:237] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:18.926617 12926 raft_consensus.cc:979] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa: : Instructing follower c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:52:18.927013 12926 raft_consensus.cc:1067] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Signalling peer c7aa762318cb4e97875333f10679a9f9 to start an election
I20260809 15:52:18.928414 11636 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "6508f17228904310be3c95818973f3a1"
dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
from {username='slave'} at 127.0.0.1:41478
I20260809 15:52:18.928917 11636 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:18.929227 11636 raft_consensus.cc:3037] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:18.937366 11010 raft_consensus.cc:2932] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:18.952978 11636 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:19.000828 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:19.001649 10843 raft_consensus.cc:3037] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:19.007898 11636 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:19.016077 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:19.016749 11013 raft_consensus.cc:3032] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:19.017138 11013 raft_consensus.cc:729] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:19.018157 11013 consensus_queue.cc:260] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:19.024552 11013 raft_consensus.cc:3037] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:19.028080 10734 catalog_manager.cc:5495] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index -1 to 4, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 3 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:19.056752 11013 raft_consensus.cc:2454] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:52:19.057821 10842 raft_consensus.cc:1261] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260809 15:52:19.057925 11015 raft_consensus.cc:1261] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260809 15:52:19.067879 13036 consensus_queue.cc:1035] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:19.069232 11604 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:19.065210 10843 raft_consensus.cc:2454] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:52:19.073776 13086 consensus_queue.cc:1035] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.004s
I20260809 15:52:19.078177 13087 ts_tablet_manager.cc:927] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615)
W20260809 15:52:19.081694 10893 consensus_peers.cc:487] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 -> Peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Couldn't send request to peer c7aa762318cb4e97875333f10679a9f9. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 06c98b3690e54b19a8bc4b4366fce35a. This is attempt 1: this message will repeat every 5th retry.
I20260809 15:52:19.075428 13060 raft_consensus.cc:2781] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:52:19.085309 13060 raft_consensus.cc:686] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:19.089311 13060 consensus_queue.cc:237] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:19.102476 13087 tablet_copy_client.cc:316] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.1:46615
I20260809 15:52:19.103272 13090 ts_tablet_manager.cc:927] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615)
I20260809 15:52:19.106830 13090 tablet_copy_client.cc:316] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.1:46615
W20260809 15:52:19.223160 13095 log.cc:924] Time spent T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.116s user 0.001s sys 0.000s
W20260809 15:52:19.238945 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 83001.9us
I20260809 15:52:19.276702 10858 tablet_copy_service.cc:140] P a53441872c0c416b8e2f26f52399f536: Received BeginTabletCopySession request for tablet 8975b3e7ddc544bc888dd61fffc5313b from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:54342)
I20260809 15:52:19.277781 10858 tablet_copy_service.cc:161] P a53441872c0c416b8e2f26f52399f536: Beginning new tablet copy session on tablet 8975b3e7ddc544bc888dd61fffc5313b from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:54342: session id = c7aa762318cb4e97875333f10679a9f9-8975b3e7ddc544bc888dd61fffc5313b
I20260809 15:52:19.279403 10857 tablet_copy_service.cc:140] P a53441872c0c416b8e2f26f52399f536: Received BeginTabletCopySession request for tablet 5e8d12b5ef9e415cbcb25e62ab228546 from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:54342)
I20260809 15:52:19.280185 10857 tablet_copy_service.cc:161] P a53441872c0c416b8e2f26f52399f536: Beginning new tablet copy session on tablet 5e8d12b5ef9e415cbcb25e62ab228546 from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:54342: session id = c7aa762318cb4e97875333f10679a9f9-5e8d12b5ef9e415cbcb25e62ab228546
I20260809 15:52:19.296195 10858 tablet_copy_source_session.cc:205] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536: Tablet Copy: opened 0 blocks and 1 log segments
W20260809 15:52:19.299824 13101 log.cc:924] Time spent T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.055s user 0.002s sys 0.000s
I20260809 15:52:19.310475 10857 tablet_copy_source_session.cc:205] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:52:19.314416 11014 raft_consensus.cc:1261] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:52:19.330132 10737 catalog_manager.cc:5495] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 1 to 2, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 2 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:19.343441 13087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8975b3e7ddc544bc888dd61fffc5313b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:52:19.350226 13090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5e8d12b5ef9e415cbcb25e62ab228546. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:52:19.385483 13086 raft_consensus.cc:2932] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:19.411516 13104 consensus_queue.cc:1035] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:52:19.445746 13107 ts_tablet_manager.cc:927] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615)
W20260809 15:52:19.481976 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 60964.7us
I20260809 15:52:19.485550 13107 tablet_copy_client.cc:316] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.1:46615
I20260809 15:52:19.510455 11015 raft_consensus.cc:2932] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:19.511713 10737 catalog_manager.cc:5495] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index -1 to 4, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New cstate: current_term: 3 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:19.526686 10930 raft_consensus.cc:1261] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:52:19.519811 13090 tablet_copy_client.cc:791] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:52:19.519585 13087 tablet_copy_client.cc:791] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:52:19.536495 13090 tablet_copy_client.cc:655] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:52:19.537482 13087 tablet_copy_client.cc:655] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:52:19.539451 10842 raft_consensus.cc:2932] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Committing config change with OpId 3.4: config changed from index -1 to 4, NON_VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) added. New config: { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } } }
I20260809 15:52:19.550330 13103 consensus_queue.cc:1035] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260809 15:52:19.597797 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 73302.7us
I20260809 15:52:19.601132 10857 tablet_copy_service.cc:140] P a53441872c0c416b8e2f26f52399f536: Received BeginTabletCopySession request for tablet 1f9b7d9e57934596b377ee8dc79ea23e from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:54342)
I20260809 15:52:19.601665 10857 tablet_copy_service.cc:161] P a53441872c0c416b8e2f26f52399f536: Beginning new tablet copy session on tablet 1f9b7d9e57934596b377ee8dc79ea23e from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:54342: session id = c7aa762318cb4e97875333f10679a9f9-1f9b7d9e57934596b377ee8dc79ea23e
I20260809 15:52:19.635818 10857 tablet_copy_source_session.cc:205] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 15:52:19.785584 10842 raft_consensus.cc:1261] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:19.823827 11015 raft_consensus.cc:1261] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
W20260809 15:52:19.866520 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 67373.5us
W20260809 15:52:19.867506 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 45971.3us
W20260809 15:52:19.878225 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 53251.3us
I20260809 15:52:19.907230 13118 ts_tablet_manager.cc:927] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9: Initiating tablet copy from peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:19.908069 13111 consensus_queue.cc:1035] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260809 15:52:20.051822 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:19.983588 13087 tablet_copy_client.cc:531] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:52:19.939584 13107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1f9b7d9e57934596b377ee8dc79ea23e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:52:19.988771 13090 tablet_copy_client.cc:531] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:52:19.988708 13111 consensus_queue.cc:1035] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.159s
I20260809 15:52:20.158068 13118 tablet_copy_client.cc:316] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9: tablet copy: Beginning tablet copy session from remote peer at address 127.9.84.2:32783
I20260809 15:52:20.161403 13090 tablet_bootstrap.cc:492] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:52:20.191514 13087 tablet_bootstrap.cc:492] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:52:20.242017 10942 tablet_copy_service.cc:140] P 20730d75ad954a82b126148f8152a9f3: Received BeginTabletCopySession request for tablet 06c98b3690e54b19a8bc4b4366fce35a from peer c7aa762318cb4e97875333f10679a9f9 ({username='slave'} at 127.0.0.1:52758)
I20260809 15:52:20.242673 10942 tablet_copy_service.cc:161] P 20730d75ad954a82b126148f8152a9f3: Beginning new tablet copy session on tablet 06c98b3690e54b19a8bc4b4366fce35a from peer c7aa762318cb4e97875333f10679a9f9 at {username='slave'} at 127.0.0.1:52758: session id = c7aa762318cb4e97875333f10679a9f9-06c98b3690e54b19a8bc4b4366fce35a
I20260809 15:52:20.265277 10942 tablet_copy_source_session.cc:205] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3: Tablet Copy: opened 0 blocks and 1 log segments
W20260809 15:52:20.346661 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 63777us
I20260809 15:52:20.358232 13107 tablet_copy_client.cc:791] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:52:20.368873 13107 tablet_copy_client.cc:655] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:52:20.437635 13090 tablet_bootstrap.cc:492] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:52:20.438566 13090 tablet_bootstrap.cc:492] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:52:20.439314 13090 ts_tablet_manager.cc:1397] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.278s user 0.032s sys 0.010s
I20260809 15:52:20.449443 13118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 06c98b3690e54b19a8bc4b4366fce35a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 15:52:20.449572 13090 raft_consensus.cc:348] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:20.450547 13090 raft_consensus.cc:729] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:52:20.451188 13090 consensus_queue.cc:260] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:20.465771 10839 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:52:20.466638 10839 raft_consensus.cc:595] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:20.469046 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260809 15:52:20.470690 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:52:20.471355 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:20.476364 13087 tablet_bootstrap.cc:492] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:52:20.484087 13087 tablet_bootstrap.cc:492] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:52:20.477280 13090 ts_tablet_manager.cc:1428] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.038s user 0.007s sys 0.000s
I20260809 15:52:20.487608 13107 tablet_copy_client.cc:531] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:52:20.491173 13118 tablet_copy_client.cc:791] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 0 data blocks...
I20260809 15:52:20.491859 13118 tablet_copy_client.cc:655] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9: tablet copy: Starting download of 1 WAL segments...
I20260809 15:52:20.496594 13107 tablet_bootstrap.cc:492] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:52:20.497816 13087 ts_tablet_manager.cc:1397] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.307s user 0.029s sys 0.008s
I20260809 15:52:20.517716 10857 tablet_copy_service.cc:342] P a53441872c0c416b8e2f26f52399f536: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-5e8d12b5ef9e415cbcb25e62ab228546 received from {username='slave'} at 127.0.0.1:54342
I20260809 15:52:20.518234 10857 tablet_copy_service.cc:433] P a53441872c0c416b8e2f26f52399f536: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-5e8d12b5ef9e415cbcb25e62ab228546 on tablet 5e8d12b5ef9e415cbcb25e62ab228546 with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:20.519804 13087 raft_consensus.cc:348] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:20.526958 13087 raft_consensus.cc:729] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:52:20.528277 13118 tablet_copy_client.cc:531] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 15:52:20.527599 13087 consensus_queue.cc:260] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:20.554334 13087 ts_tablet_manager.cc:1428] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.056s user 0.005s sys 0.002s
I20260809 15:52:20.568399 10857 tablet_copy_service.cc:342] P a53441872c0c416b8e2f26f52399f536: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-8975b3e7ddc544bc888dd61fffc5313b received from {username='slave'} at 127.0.0.1:54342
I20260809 15:52:20.568801 10857 tablet_copy_service.cc:433] P a53441872c0c416b8e2f26f52399f536: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-8975b3e7ddc544bc888dd61fffc5313b on tablet 8975b3e7ddc544bc888dd61fffc5313b with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:20.570188 13118 tablet_bootstrap.cc:492] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9: Bootstrap starting.
I20260809 15:52:20.612274 13107 tablet_bootstrap.cc:492] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:52:20.613157 13107 tablet_bootstrap.cc:492] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:52:20.613866 13107 ts_tablet_manager.cc:1397] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.117s user 0.029s sys 0.001s
I20260809 15:52:20.625303 13118 tablet_bootstrap.cc:492] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=0 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 15:52:20.626284 13118 tablet_bootstrap.cc:492] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9: Bootstrap complete.
I20260809 15:52:20.626993 13118 ts_tablet_manager.cc:1397] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9: Time spent bootstrapping tablet: real 0.057s user 0.023s sys 0.005s
I20260809 15:52:20.632352 13107 raft_consensus.cc:348] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:20.633354 13107 raft_consensus.cc:729] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:52:20.634019 13107 consensus_queue.cc:260] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:20.629977 13118 raft_consensus.cc:348] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 3 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:20.640863 13118 raft_consensus.cc:729] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 3 LEARNER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Initialized, Role: LEARNER
I20260809 15:52:20.641520 13118 consensus_queue.cc:260] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:20.643277 13107 ts_tablet_manager.cc:1428] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.029s user 0.005s sys 0.001s
I20260809 15:52:20.646430 13118 ts_tablet_manager.cc:1428] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9: Time spent starting tablet: real 0.019s user 0.008s sys 0.000s
I20260809 15:52:20.649245 10942 tablet_copy_service.cc:342] P 20730d75ad954a82b126148f8152a9f3: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-06c98b3690e54b19a8bc4b4366fce35a received from {username='slave'} at 127.0.0.1:52758
I20260809 15:52:20.649597 10942 tablet_copy_service.cc:433] P 20730d75ad954a82b126148f8152a9f3: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-06c98b3690e54b19a8bc4b4366fce35a on tablet 06c98b3690e54b19a8bc4b4366fce35a with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:20.652509 11635 raft_consensus.cc:1203] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEARNER]: Deduplicated request from leader. Original: 4.4->[4.5-4.5] Dedup: 4.5->[]
I20260809 15:52:20.653216 10857 tablet_copy_service.cc:342] P a53441872c0c416b8e2f26f52399f536: Request end of tablet copy session c7aa762318cb4e97875333f10679a9f9-1f9b7d9e57934596b377ee8dc79ea23e received from {username='slave'} at 127.0.0.1:54342
I20260809 15:52:20.653640 10857 tablet_copy_service.cc:433] P a53441872c0c416b8e2f26f52399f536: ending tablet copy session c7aa762318cb4e97875333f10679a9f9-1f9b7d9e57934596b377ee8dc79ea23e on tablet 1f9b7d9e57934596b377ee8dc79ea23e with peer c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:20.783563 11635 raft_consensus.cc:1203] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.1->[1.2-1.2] Dedup: 1.2->[]
I20260809 15:52:20.817329 13131 raft_consensus.cc:979] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536: : Instructing follower 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:20.817986 12933 raft_consensus.cc:1067] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Signalling peer 20730d75ad954a82b126148f8152a9f3 to start an election
I20260809 15:52:20.822513 10923 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:54224
I20260809 15:52:20.823153 10923 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:20.823982 10923 raft_consensus.cc:3037] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:20.832322 11635 raft_consensus.cc:1203] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4] Dedup: 3.4->[]
W20260809 15:52:20.850862 10719 catalog_manager.cc:5025] ChangeConfig:REMOVE_PEER RPC for tablet 0b2b0df042e44725a7c91deae3df1956 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223) due to error UNKNOWN_ERROR; will retry: Invalid argument: Cannot remove peer d74944fa9fa14ec78b0640ea709ff3aa from the config because it is the leader. Force another leader to be elected to remove this peer. Consensus state: current_term: 6 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:20.973419 10923 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:21.014555 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:21.015321 10837 raft_consensus.cc:3032] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:21.015666 10837 raft_consensus.cc:729] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:21.016345 10837 consensus_queue.cc:260] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:21.017436 10837 raft_consensus.cc:3037] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:21.018816 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:21.019431 11009 raft_consensus.cc:3037] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:21.027151 11635 raft_consensus.cc:1203] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 3 LEARNER]: Deduplicated request from leader. Original: 3.3->[3.4-3.4] Dedup: 3.4->[]
I20260809 15:52:21.045307 11009 raft_consensus.cc:2454] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:21.067188 10891 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:21.068068 13132 raft_consensus.cc:2781] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:21.016494 10923 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:21.068604 13132 raft_consensus.cc:686] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:21.069804 10837 raft_consensus.cc:2454] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:21.069731 13132 consensus_queue.cc:237] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:21.104526 10736 catalog_manager.cc:5495] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 3 to 4, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 4 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:21.174913 13131 raft_consensus.cc:1050] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:52:21.177711 13131 consensus_queue.cc:237] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:21.211500 11635 raft_consensus.cc:1261] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEARNER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:21.213812 11013 raft_consensus.cc:1261] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:21.214046 10923 raft_consensus.cc:1261] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:21.216066 13128 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:21.217377 12933 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:21.223156 13131 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.002s
I20260809 15:52:21.251498 13131 raft_consensus.cc:2932] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Committing config change with OpId 4.6: config changed from index 5 to 6, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:21.260036 10923 raft_consensus.cc:2932] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:21.356490 11637 raft_consensus.cc:2932] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:21.463682 10734 catalog_manager.cc:5495] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 reported cstate change: config changed from index 5 to 6, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 4 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:22.808377 10843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45578 (request call id 1867) took 1255 ms. Trace:
W20260809 15:52:22.920570 11044 debug-util.cc:398] Leaking SignalData structure 0x7b0800174ac0 after lost signal to thread 9556
I20260809 15:52:23.049467 10843 rpcz_store.cc:276] 0809 15:52:21.552390 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:22.806790 (+1254400us) service_pool.cc:227] Handling call
0809 15:52:22.807520 (+ 730us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:22.807842 (+ 322us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:22.807880 (+ 38us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:22.807929 (+ 49us) raft_consensus.cc:1631] Marking committed up to 1
0809 15:52:22.807962 (+ 33us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:22.808030 (+ 68us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:22.808184 (+ 154us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:22.984865 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:23.225342 11013 raft_consensus.cc:2932] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Committing config change with OpId 4.6: config changed from index 5 to 6, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:23.321100 13173 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:23.321640 13175 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.265892 13174 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:23.321913 13173 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.322315 13175 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.322904 13174 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.324265 13173 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.324812 13173 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.325068 13175 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.325598 13175 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.325304 13173 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.351845 13158 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:23.325613 13174 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:23.352545 13174 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.352372 13158 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.353026 13174 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.353943 13173 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.354367 13158 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.354463 13173 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:23.327858 13177 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:23.328409 13176 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.354939 13173 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.355468 13176 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.357172 13173 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.357347 13176 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.358011 13176 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:23.358479 13176 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.360348 13176 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.335692 11016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 1845) took 1863 ms. Trace:
I20260809 15:52:23.360759 13173 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:23.328838 13178 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.360940 11016 rpcz_store.cc:276] 0809 15:52:21.471898 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:23.334777 (+1862879us) service_pool.cc:227] Handling call
0809 15:52:23.335101 (+ 324us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:23.335263 (+ 162us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:23.335288 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:23.335326 (+ 38us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:52:23.335358 (+ 32us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:23.335420 (+ 62us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:23.335524 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:23.362269 13164 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.361668 13178 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.362741 13164 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.364459 13178 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:23.364698 13164 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.326051 13175 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.365213 13164 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.361330 13173 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.340938 13060 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.378934 13060 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.380914 13060 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.381379 13060 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.341075 13172 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:23.382340 13172 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.384409 13172 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.384688 13132 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.385392 13105 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:23.385346 13132 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.386107 13105 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.388111 13132 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.388696 13132 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.388813 13105 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.389400 13105 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.389202 13132 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.389911 13105 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.391758 13105 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.392165 13105 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.392554 13105 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.382380 13121 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
W20260809 15:52:23.394671 10872 debug-util.cc:398] Leaking SignalData structure 0x7b08001c9120 after lost signal to thread 9556
I20260809 15:52:23.381791 13060 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:23.395682 10872 debug-util.cc:398] Leaking SignalData structure 0x7b08000a6ae0 after lost signal to thread 10773
I20260809 15:52:23.396328 13060 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.396446 13172 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:52:23.396890 10872 debug-util.cc:398] Leaking SignalData structure 0x7b08001fca20 after lost signal to thread 10875
W20260809 15:52:23.397183 10872 debug-util.cc:398] Leaking SignalData structure 0x7b08001fca00 after lost signal to thread 10877
I20260809 15:52:23.397552 13060 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.397095 13172 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.397998 13060 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.400063 13176 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:23.401096 13060 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:23.347883 10893 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.804s user 0.017s sys 0.017s
W20260809 15:52:23.401602 10893 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.857s user 0.017s sys 0.017s
I20260809 15:52:23.401911 13105 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.344844 13181 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:23.354954 13174 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.402817 13174 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.403329 13174 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.402937 13132 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.355412 13177 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.335916 13180 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:23.407964 13180 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.338369 13179 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.365139 13178 raft_consensus.cc:482] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:23.410045 13180 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.410923 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:23.410354 13179 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.413163 13179 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.413611 13179 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:23.410591 13178 raft_consensus.cc:504] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.415894 13178 leader_election.cc:288] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.366989 13175 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.405952 13174 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.365868 13164 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.400688 13176 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.417951 13164 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.394687 13121 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.418131 13176 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.407546 13177 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.407356 13172 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.419688 13121 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.414059 13179 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:23.354876 13158 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.421015 13158 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.421509 13179 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.422135 13181 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.423002 13158 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.425088 13167 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:23.425590 13167 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.427593 13167 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.428150 13173 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.401607 13060 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:23.429111 13060 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.429500 13127 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:23.430032 13127 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:23.438745 10872 debug-util.cc:398] Leaking SignalData structure 0x7b0800053d80 after lost signal to thread 10959
W20260809 15:52:23.439257 10872 debug-util.cc:398] Leaking SignalData structure 0x7b080034f660 after lost signal to thread 10960
W20260809 15:52:23.445585 10890 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.895s user 0.012s sys 0.018s
W20260809 15:52:23.445919 10890 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.909s user 0.012s sys 0.018s
I20260809 15:52:23.445984 13060 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.451612 13158 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.452174 13158 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.453122 13173 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:23.455168 13179 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.453613 13173 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.462553 13181 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.462690 13177 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.455762 13179 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.463160 13177 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.465334 13127 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.467532 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:23.468197 10847 raft_consensus.cc:2454] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 1.
I20260809 15:52:23.468276 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:23.468916 10839 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:23.469825 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:23.470371 10844 raft_consensus.cc:2454] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:23.470830 13173 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.454015 13158 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.473572 10837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 481) took 2118 ms. Trace:
I20260809 15:52:23.474872 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:23.475832 13121 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.476465 13121 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.474025 10837 rpcz_store.cc:276] 0809 15:52:21.355173 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:23.359361 (+2004188us) service_pool.cc:227] Handling call
0809 15:52:23.359663 (+ 302us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:23.388338 (+ 28675us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:23.388384 (+ 46us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:23.388423 (+ 39us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:23.388454 (+ 31us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:23.473297 (+ 84843us) spinlock_profiling.cc:243] Waited 28.5 ms on lock 0x7b5400005fd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:23.473363 (+ 66us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:23.473506 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":65517824}
I20260809 15:52:23.479048 13177 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.480329 13179 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.481597 13121 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:23.482132 13164 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.492962 13128 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:23.493436 13128 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.494472 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:23.495829 13128 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.496492 13128 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.497031 13128 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.499482 13128 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.500587 13128 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:23.501160 13128 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.502694 10734 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33038 (request call id 388) took 2163 ms. Trace:
I20260809 15:52:23.505883 10736 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33054 (request call id 390) took 2184 ms. Trace:
I20260809 15:52:23.503016 10734 rpcz_store.cc:276] 0809 15:52:21.339472 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:21.340363 (+ 891us) service_pool.cc:227] Handling call
0809 15:52:23.502611 (+2162248us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 15:52:21.895634 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 15:52:21.896096 (+ 462us) write_op.cc:432] Acquiring schema lock in shared mode
0809 15:52:21.896127 (+ 31us) write_op.cc:435] Acquired schema lock
0809 15:52:21.896141 (+ 14us) tablet.cc:660] Decoding operations
0809 15:52:21.896457 (+ 316us) write_op.cc:620] Acquiring the partition lock for write op
0809 15:52:21.896511 (+ 54us) write_op.cc:641] Partition lock acquired for write op
0809 15:52:21.896540 (+ 29us) tablet.cc:683] Acquiring locks for 1 operations
0809 15:52:21.896703 (+ 163us) tablet.cc:699] Row locks acquired
0809 15:52:21.896717 (+ 14us) write_op.cc:260] PREPARE: finished
0809 15:52:21.896818 (+ 101us) write_op.cc:270] Start()
0809 15:52:21.896887 (+ 69us) write_op.cc:276] Timestamp: P: 1786290741896788 usec, L: 0
0809 15:52:21.896900 (+ 13us) op_driver.cc:331] REPLICATION: starting
0809 15:52:21.897389 (+ 489us) log.cc:842] Serialized 587 byte log entry
0809 15:52:21.902637 (+ 5248us) op_driver.cc:444] REPLICATION: finished
0809 15:52:23.369782 (+1467145us) write_op.cc:301] APPLY: starting
0809 15:52:23.369886 (+ 104us) tablet.cc:1365] starting BulkCheckPresence
0809 15:52:23.370016 (+ 130us) tablet.cc:1368] finished BulkCheckPresence
0809 15:52:23.370187 (+ 171us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 15:52:23.371080 (+ 893us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 15:52:23.371265 (+ 185us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 15:52:23.371330 (+ 65us) write_op.cc:312] APPLY: finished
0809 15:52:23.371568 (+ 238us) log.cc:842] Serialized 23 byte log entry
0809 15:52:23.486350 (+114782us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:52:23.488116 (+ 1766us) write_op.cc:454] Released schema lock
0809 15:52:23.488161 (+ 45us) write_op.cc:341] FINISH: Updating metrics
0809 15:52:23.488452 (+ 291us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:52:23.488482 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":1466864,"apply.run_cpu_time_us":18699,"apply.run_wall_time_us":118962,"num_ops":1,"prepare.queue_time_us":428645,"prepare.run_cpu_time_us":2020,"prepare.run_wall_time_us":2020,"replication_time_us":5642,"thread_start_us":86241,"threads_started":1,"wal-append.queue_time_us":83271}]]}
I20260809 15:52:23.503801 13128 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:23.513590 10891 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.975s user 0.015s sys 0.012s
W20260809 15:52:23.513923 10891 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.975s user 0.015s sys 0.012s
W20260809 15:52:23.522001 10872 debug-util.cc:398] Leaking SignalData structure 0x7b08002e1c40 after lost signal to thread 11047
I20260809 15:52:23.506235 10736 rpcz_store.cc:276] 0809 15:52:21.321306 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:21.363037 (+ 41731us) service_pool.cc:227] Handling call
0809 15:52:23.505826 (+2142789us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":100968,"reported_tablets":1}
W20260809 15:52:23.528395 10872 debug-util.cc:398] Leaking SignalData structure 0x7b0800344c00 after lost signal to thread 11667
I20260809 15:52:23.538260 12933 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:23.539089 13184 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.539798 13184 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.542649 13184 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.486626 13167 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.487895 13060 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.529250 13132 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.555493 13132 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.555570 13167 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:23.536000 13127 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.544282 13175 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.527366 13164 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.555496 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:23.543965 13180 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.546823 13183 raft_consensus.cc:482] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:23.558327 13180 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.547518 13181 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:23.548323 13131 raft_consensus.cc:482] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.548763 13177 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.549162 13178 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.550220 13179 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:23.545238 12933 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.551011 13111 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:23.554919 13174 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.584810 13162 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.584949 13161 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.549949 13128 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:23.585132 13174 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.585518 13162 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.594250 13169 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.596838 10848 raft_consensus.cc:1261] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:52:23.594067 13128 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.577924 13060 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.558717 13183 raft_consensus.cc:504] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.595953 12933 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.622745 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:23.623261 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:23.623701 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:23.623972 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:23.624269 10844 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:52:23.624617 10843 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:23.625659 13193 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.594828 13161 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.628943 13194 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.629123 13161 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.582763 13181 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.586107 13179 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.583911 13131 raft_consensus.cc:504] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.583386 13178 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.626487 13193 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.557164 13175 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.585757 13177 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.629671 13194 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:23.773864 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 146600us
I20260809 15:52:23.556947 13127 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.772928 13185 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.590974 13111 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.802600 13132 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.801213 13187 raft_consensus.cc:482] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.819869 13179 leader_election.cc:288] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.820920 13132 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:23.822279 13175 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:23.823943 13177 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.824749 13178 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:23.825840 13060 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.827302 13195 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:23.617316 13169 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.837996 13132 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.844273 13174 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.844606 13167 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.847630 13162 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.851016 13164 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.864061 13167 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.852360 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:23.865000 13194 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.854486 13128 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.858193 13193 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.868575 13197 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.868718 13180 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.869395 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:23.859593 13183 leader_election.cc:288] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.863289 13131 leader_election.cc:288] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.866971 13127 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.885021 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:23.871059 13168 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.870523 13181 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.815191 13185 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:23.885885 11639 raft_consensus.cc:2454] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:23.841163 13187 raft_consensus.cc:504] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.907832 13183 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.908568 13183 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.910287 13187 leader_election.cc:288] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.907912 13205 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:23.870875 13167 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.887928 13111 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:23.846429 13195 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.919021 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:23.920161 13166 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:23.915370 13127 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:23.928735 10927 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:23.929286 10927 raft_consensus.cc:595] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:23.920400 13132 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.927327 13207 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:23.932722 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:23.928330 13195 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.931353 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:23.916290 13183 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:23.927345 13160 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:23.918356 13169 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.937806 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:23.885929 13168 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.940961 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:23.942817 10923 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:23.941898 11013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 1842) took 2467 ms. Trace:
I20260809 15:52:23.944191 11013 rpcz_store.cc:276] 0809 15:52:21.470998 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:23.224641 (+1753643us) service_pool.cc:227] Handling call
0809 15:52:23.225021 (+ 380us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:23.225225 (+ 204us) raft_consensus.cc:1520] Early marking committed up to index 6
0809 15:52:23.340333 (+115108us) log.cc:842] Serialized 14 byte log entry
0809 15:52:23.938114 (+597781us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:23.938161 (+ 47us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:23.938202 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:23.938262 (+ 60us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:23.938357 (+ 95us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":529967,"raft.queue_time_us":543892,"raft.run_cpu_time_us":98,"raft.run_wall_time_us":98,"thread_start_us":63378,"threads_started":1}
I20260809 15:52:23.947561 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:23.913626 13205 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.935334 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:23.874715 13197 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:23.948598 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 99272.3us
I20260809 15:52:23.954643 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:23.956877 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:23.957074 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:23.957413 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:23.957513 10848 raft_consensus.cc:2454] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 1.
I20260809 15:52:23.958505 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:23.960270 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:23.963266 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:23.935299 13127 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.964004 10848 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:23.935438 13166 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.965402 13198 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:23.962381 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:23.966044 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:23.974745 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:23.967224 13205 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:23.967664 13099 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:23.968109 13208 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.942412 13133 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:23.964640 10927 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:23.974463 11016 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:23.984674 13165 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:23.948678 13160 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:23.958128 13192 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:23.994062 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:23.994796 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:23.996337 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:23.941248 13207 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.996697 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:24.012995 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:24.007700 10893 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:24.014278 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:23.986119 13132 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:24.012719 11010 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:23.999449 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.016474 10893 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:24.020896 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.021700 13166 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:24.021739 11606 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:24.022441 13202 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:24.022513 13169 raft_consensus.cc:2781] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:24.022646 13199 raft_consensus.cc:2781] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:52:24.024434 10844 raft_consensus.cc:2454] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:24.007339 13127 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:24.025067 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.007076 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.036477 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.024057 11606 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:24.040987 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:23.994724 13133 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 2 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:23.994972 13195 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:24.024652 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:23.995297 13183 raft_consensus.cc:482] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:24.006846 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.047117 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.051753 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.054243 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.051746 13180 raft_consensus.cc:2781] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:24.054833 10843 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:24.051357 11606 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:24.057181 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.057416 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.059685 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.057056 13180 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:24.043619 13111 raft_consensus.cc:2781] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:52:24.041914 13169 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:24.044554 13166 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:24.034754 13185 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:24.078841 10893 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:24.070289 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.049897 13167 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:24.052156 13199 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:24.023845 13132 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.007666 13208 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:24.063107 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.006222 13198 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:23.997061 13099 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.009629 13192 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:24.089457 11639 raft_consensus.cc:2454] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:24.221956 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.196586 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.231779 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.005635 13165 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:24.114544 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.197762 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.233035 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.062987 10921 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:24.215896 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.236018 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:24.178208 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 99503.9us
I20260809 15:52:24.052242 13202 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
W20260809 15:52:24.238624 11044 debug-util.cc:398] Leaking SignalData structure 0x7b08001af000 after lost signal to thread 10773
I20260809 15:52:24.195389 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:24.239035 11044 debug-util.cc:398] Leaking SignalData structure 0x7b08001426c0 after lost signal to thread 10789
W20260809 15:52:24.239964 11044 debug-util.cc:398] Leaking SignalData structure 0x7b08001afe60 after lost signal to thread 10875
W20260809 15:52:24.240147 11044 debug-util.cc:398] Leaking SignalData structure 0x7b0800201680 after lost signal to thread 10877
W20260809 15:52:24.240830 11044 debug-util.cc:398] Leaking SignalData structure 0x7b0800353700 after lost signal to thread 10959
W20260809 15:52:24.240991 11044 debug-util.cc:398] Leaking SignalData structure 0x7b08000fea80 after lost signal to thread 10960
I20260809 15:52:24.242822 11606 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:24.056128 13168 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:24.064881 13180 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:24.195135 13203 raft_consensus.cc:482] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:24.247041 13192 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:24.079206 13169 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:24.065557 13111 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:24.196779 13185 raft_consensus.cc:2781] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:24.214641 13215 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:24.065271 13187 raft_consensus.cc:2781] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:24.235621 13199 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:24.252610 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.197016 13212 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:24.254487 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.253319 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.257282 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.237341 13209 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:24.258200 11606 leader_election.cc:302] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:24.066452 13183 raft_consensus.cc:504] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.250267 13178 raft_consensus.cc:2781] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:52:24.250286 13214 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:24.214430 13200 consensus_queue.cc:1035] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.249s
I20260809 15:52:24.078017 13195 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.251623 13197 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:24.266155 10841 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:24.265832 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.260500 13111 raft_consensus.cc:3037] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:24.113936 13166 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:24.287499 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.258772 13216 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:24.263154 13185 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:24.281126 13178 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:24.264454 13187 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
W20260809 15:52:24.291649 11044 debug-util.cc:398] Leaking SignalData structure 0x7b080026a0e0 after lost signal to thread 11047
I20260809 15:52:24.309588 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:24.310240 11044 debug-util.cc:398] Leaking SignalData structure 0x7b0800332c00 after lost signal to thread 11667
I20260809 15:52:24.281594 13177 raft_consensus.cc:2781] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:24.292696 10929 raft_consensus.cc:2454] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:52:24.293217 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.286444 13209 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:24.260502 13192 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:24.300783 13221 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:24.302143 13099 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:24.303040 13208 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:24.304301 13165 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:24.313334 13178 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:24.313141 13187 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:24.311743 13198 leader_election.cc:288] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:24.316905 10893 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:24.282517 13212 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:24.308844 13185 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:24.327849 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.266032 13203 raft_consensus.cc:504] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.330847 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.312757 13177 raft_consensus.cc:482] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:24.293604 13207 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:24.329643 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.289505 13215 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.319085 13133 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:24.340360 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.326375 13217 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:24.341336 10931 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:24.344257 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.344525 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.347069 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.347899 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.348212 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.350493 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.335734 13194 raft_consensus.cc:2781] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:24.338016 13177 raft_consensus.cc:3037] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:24.332427 13221 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.356732 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.327008 13192 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:24.307895 13214 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.367233 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.369135 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.358994 13128 raft_consensus.cc:1050] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:52:24.371884 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.359122 13132 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:24.358280 13181 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:24.359606 13194 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:24.384756 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.385413 11635 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:24.386765 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.310482 13216 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.385823 10893 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:24.396205 10893 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:24.373025 13208 raft_consensus.cc:482] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:24.399776 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.399930 10893 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:24.405707 10893 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:24.385468 13194 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:24.407809 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.407855 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.408484 13197 raft_consensus.cc:2726] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:24.408401 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.409499 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.410018 13166 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:24.362439 13217 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.410385 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.388391 13195 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:24.377907 13133 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:24.413652 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.393633 13183 leader_election.cc:288] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:24.402514 13132 raft_consensus.cc:2726] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:24.450034 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.399407 13200 raft_consensus.cc:2781] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:24.469097 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.389421 13181 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:24.444988 13202 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:24.475687 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.412125 13209 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:24.485633 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 84541.2us
I20260809 15:52:24.499574 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.489569 11606 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:24.464409 13218 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:24.444978 13208 raft_consensus.cc:504] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.508682 10893 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:24.510561 11606 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:24.512181 10893 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:24.491833 13196 raft_consensus.cc:2726] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:24.513757 10893 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:24.513798 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.514232 11606 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:24.520820 11606 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:24.520599 10893 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:24.523536 10893 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:24.506325 13200 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:24.495287 10922 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:24.498579 10921 raft_consensus.cc:2454] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:24.511742 13060 raft_consensus.cc:2726] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:24.553525 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.556545 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.546554 13183 raft_consensus.cc:2726] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:24.515327 13105 raft_consensus.cc:2726] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:24.558853 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.522236 13229 raft_consensus.cc:2781] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:24.563946 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.519522 13168 raft_consensus.cc:2726] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:24.549741 13200 raft_consensus.cc:3037] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
W20260809 15:52:24.571918 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 50119us
I20260809 15:52:24.550624 13195 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:24.510880 13133 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:24.549738 13212 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:24.574950 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.575413 11606 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:24.560286 13192 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:24.558704 13132 raft_consensus.cc:2726] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:24.575049 13195 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:24.582151 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.585011 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:24.585287 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:24.584086 13133 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:24.565296 13207 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:24.590963 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:24.590463 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:24.571183 13060 raft_consensus.cc:2726] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:24.593357 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:24.597254 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.606583 11010 raft_consensus.cc:2454] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 1.
I20260809 15:52:24.584247 13160 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:24.579842 13229 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:24.610144 10893 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:24.584573 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.595464 13195 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:24.574244 13221 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:24.578727 13203 leader_election.cc:288] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:24.581153 13215 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:24.624531 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:24.519131 13218 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.625902 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.594724 13214 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:24.614358 13229 raft_consensus.cc:3037] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:24.622026 13162 raft_consensus.cc:2726] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:24.586993 13128 consensus_queue.cc:237] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 0, Current term: 1, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.640666 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.642536 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.633491 13208 leader_election.cc:288] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:24.648691 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:24.609565 13207 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.653679 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:24.654035 13228 log.cc:924] Time spent T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.132s user 0.001s sys 0.000s
I20260809 15:52:24.654611 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.656200 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.661386 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.662645 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.663335 10924 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:24.670199 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:24.670598 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.673748 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.676154 13233 raft_consensus.cc:2726] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:24.680837 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.680912 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.683665 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:24.683905 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 105861us
I20260809 15:52:24.664899 13181 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:24.656482 13217 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:24.649129 13216 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:24.698112 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:24.706745 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.651613 13180 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.720369 11637 raft_consensus.cc:1226] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:24.723906 10893 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:24.727509 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:24.731757 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.733193 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.697417 13230 raft_consensus.cc:2781] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:24.735313 10893 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
W20260809 15:52:24.736974 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 139255us
I20260809 15:52:24.737689 11606 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:24.738147 13211 raft_consensus.cc:2726] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:24.739207 11606 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:24.740402 11606 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:24.734491 13225 raft_consensus.cc:1050] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:52:24.737236 13105 raft_consensus.cc:1050] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:52:24.746803 10736 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33038 (request call id 389) took 1186 ms. Trace:
I20260809 15:52:24.738544 13233 raft_consensus.cc:2726] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:24.755167 10736 rpcz_store.cc:276] 0809 15:52:23.554975 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:23.557567 (+ 2592us) service_pool.cc:227] Handling call
0809 15:52:24.741095 (+1183528us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":2}
I20260809 15:52:24.730209 13216 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:24.739096 13197 raft_consensus.cc:2726] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:24.760792 13232 raft_consensus.cc:2781] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:24.744349 13230 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:24.768502 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:24.770267 13232 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:24.770898 13232 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:24.777596 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.764729 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:24.785208 10838 raft_consensus.cc:2454] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:24.753048 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:24.792980 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:24.793813 11016 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
W20260809 15:52:24.795326 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 233790us
I20260809 15:52:24.800894 13111 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.801559 13207 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:24.802214 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.803298 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.762130 13211 consensus_queue.cc:1046] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260809 15:52:24.803464 13185 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.804898 13211 raft_consensus.cc:3032] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:24.805245 13211 raft_consensus.cc:729] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:24.805349 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.806183 13211 consensus_queue.cc:260] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.808140 13211 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:24.765085 13216 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:24.800933 13169 consensus_meta.cc:287] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.539s user 0.003s sys 0.008s
I20260809 15:52:24.811349 13169 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:24.814219 10929 raft_consensus.cc:1226] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:24.815950 13169 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:24.769491 13230 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:24.777570 13227 raft_consensus.cc:2781] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260809 15:52:24.797914 13199 consensus_meta.cc:287] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.516s user 0.013s sys 0.006s
I20260809 15:52:24.821390 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:24.785578 13225 consensus_queue.cc:237] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.899334 10978 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:24.828917 13180 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:24.926156 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.918295 13208 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:24.928738 13208 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.929128 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:24.931710 13216 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:24.931910 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:24.937655 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:24.829991 13199 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:24.940685 13199 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:24.941839 10931 raft_consensus.cc:1226] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:24.944764 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:24.944581 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:52:24.907579 10811 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.84.2:32783 timed out after 1.521s (SENT) [suppressed 4 similar messages]
I20260809 15:52:24.821440 13187 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:24.896138 13227 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:24.958242 10893 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:24.998265 13227 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:24.905951 13218 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:25.013146 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 54983.2us
I20260809 15:52:24.931715 13208 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:24.943511 13223 raft_consensus.cc:2781] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:24.905030 13185 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:25.013651 13177 consensus_meta.cc:287] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.655s user 0.010s sys 0.007s
I20260809 15:52:25.032763 13132 raft_consensus.cc:2726] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:25.034292 13223 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:25.142598 11009 raft_consensus.cc:1261] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:52:25.192209 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.189391 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:25.194351 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 251035us
I20260809 15:52:25.079403 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.195413 11016 raft_consensus.cc:2454] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 1.
I20260809 15:52:25.142493 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.189862 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:25.202105 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:25.205380 10844 consensus_queue.cc:237] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:25.207304 13187 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:25.212049 13194 consensus_meta.cc:287] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.717s user 0.009s sys 0.004s
W20260809 15:52:25.209537 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 301955us
I20260809 15:52:25.214872 13194 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:25.172582 13181 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:25.216602 13181 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:25.221596 13181 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:25.223825 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.224436 11016 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:25.044256 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:25.225488 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.227083 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260809 15:52:25.187076 13111 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:25.229596 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.231220 11013 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 1.
I20260809 15:52:25.231370 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.217842 13194 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:25.225965 13178 consensus_meta.cc:287] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.891s user 0.000s sys 0.021s
I20260809 15:52:25.240595 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:25.241696 10923 raft_consensus.cc:2379] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 4: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:25.208099 13223 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:25.076143 13177 raft_consensus.cc:504] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:25.258045 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 63104.1us
I20260809 15:52:25.235981 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:25.221864 13232 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:25.260429 13229 consensus_meta.cc:287] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.621s user 0.004s sys 0.008s
I20260809 15:52:25.317859 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:25.330711 11640 raft_consensus.cc:1226] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:25.296496 10838 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:25.247920 13178 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:25.379761 13200 consensus_meta.cc:287] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.793s user 0.005s sys 0.008s
I20260809 15:52:25.380259 13200 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:25.380824 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:25.383728 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:52:25.384599 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:25.386613 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:52:25.408090 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 111375us
I20260809 15:52:25.432826 10737 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33054 (request call id 391) took 1001 ms. Trace:
I20260809 15:52:25.334245 13177 leader_election.cc:288] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:25.483958 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:25.522683 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 233534us
W20260809 15:52:25.526933 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 518487us
I20260809 15:52:25.321678 13229 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:25.540010 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 294226us
I20260809 15:52:25.428030 10927 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:25.433372 10737 rpcz_store.cc:276] 0809 15:52:24.381824 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:24.388163 (+ 6339us) service_pool.cc:227] Handling call
0809 15:52:25.383621 (+995458us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":1}
I20260809 15:52:25.588189 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.586689 10978 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
W20260809 15:52:25.589022 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 160277us
I20260809 15:52:25.588892 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.608270 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.608907 11014 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:25.607339 10978 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:25.610836 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.611446 11018 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:25.612099 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.612995 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.613209 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.615455 10891 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:25.614230 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.616601 10891 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:25.606173 10811 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:25.618552 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.614161 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.620788 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.622455 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.622668 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.623054 11011 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:25.604868 13099 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:25.619694 11012 raft_consensus.cc:2454] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
W20260809 15:52:25.615717 13195 consensus_meta.cc:287] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.976s user 0.012s sys 0.008s
I20260809 15:52:25.629981 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.630764 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.618219 13217 raft_consensus.cc:2726] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:25.624768 13191 raft_consensus.cc:2781] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:25.623631 13099 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:25.613759 13223 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:25.664804 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:25.665503 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:25.648006 10811 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9
W20260809 15:52:25.687105 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 69023.6us
I20260809 15:52:25.688037 10891 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:25.646523 13232 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:25.689002 11009 raft_consensus.cc:2379] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 4: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
W20260809 15:52:25.643728 13230 consensus_meta.cc:287] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.814s user 0.006s sys 0.004s
I20260809 15:52:25.689642 13230 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:25.692090 13230 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:25.640614 13247 raft_consensus.cc:2781] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:25.694767 13247 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:25.695070 13223 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:25.636986 13200 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:25.695951 13200 raft_consensus.cc:2726] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 15:52:25.648236 13227 consensus_meta.cc:287] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.649s user 0.008s sys 0.004s
I20260809 15:52:25.697027 13227 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:25.636488 13195 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:25.698655 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 50714us
I20260809 15:52:25.699421 13167 consensus_queue.cc:1035] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:52:25.637652 13235 raft_consensus.cc:2726] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:25.700922 10811 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:25.695137 13247 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:25.631800 13185 raft_consensus.cc:2781] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:25.702611 10811 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
W20260809 15:52:25.704583 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 79060.9us
I20260809 15:52:25.674099 13178 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:25.705755 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 98888.7us
I20260809 15:52:25.700805 13195 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:25.650841 13229 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:25.656020 13246 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:25.678619 13191 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:25.698709 10891 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:25.716043 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:25.716703 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:25.705238 13181 raft_consensus.cc:2781] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:25.732069 10891 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:25.737083 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54236 (request call id 1884) took 1431 ms. Trace:
I20260809 15:52:25.713438 13185 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:25.739009 13185 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:25.737449 10931 rpcz_store.cc:276] 0809 15:52:24.294833 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:24.381496 (+ 86663us) service_pool.cc:227] Handling call
0809 15:52:24.382189 (+ 693us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:24.946414 (+564225us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:25.725819 (+779405us) spinlock_profiling.cc:243] Waited 559 ms on lock 0x7b540025b7d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:25.726025 (+ 206us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1286660864}
I20260809 15:52:25.751309 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:25.707218 13249 raft_consensus.cc:2726] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:25.706837 10838 raft_consensus.cc:2454] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:25.723158 11634 raft_consensus.cc:1226] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:25.734673 13191 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:25.737529 13181 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:25.755734 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:25.767657 11606 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:25.766132 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:25.756600 13255 raft_consensus.cc:2726] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:25.772140 13259 raft_consensus.cc:2726] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:25.761513 13181 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:25.759897 13254 raft_consensus.cc:2781] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:25.782264 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.737196 13246 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:25.753492 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:25.779863 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.780495 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.781533 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.782549 13254 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:25.784744 13254 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:25.783748 11015 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:25.786578 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.787578 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.788424 13259 raft_consensus.cc:1013] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: attempt to promote peer c7aa762318cb4e97875333f10679a9f9: there is already a config change operation in progress. Unable to promote follower until it completes. Doing nothing.
I20260809 15:52:25.788756 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.789438 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.789522 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:25.784135 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 52190.7us
W20260809 15:52:25.792111 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 184861us
I20260809 15:52:25.793367 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:25.793459 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.794060 10837 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:25.794776 11604 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:25.795348 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:25.795678 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 93156.4us
I20260809 15:52:25.796882 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:25.796746 10893 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:25.799146 11604 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:25.800238 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:25.802891 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:25.804507 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:25.808341 10891 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:25.809096 13255 raft_consensus.cc:2726] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:25.810042 13178 raft_consensus.cc:2726] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:25.811131 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:25.812280 11604 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:25.813004 13178 raft_consensus.cc:2726] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:25.809480 10891 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:25.813987 13255 raft_consensus.cc:2726] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:25.813716 13247 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:25.815697 13247 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:25.815665 10891 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:25.816748 13197 raft_consensus.cc:2726] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 15:52:25.817129 13227 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:25.820420 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:25.821395 13175 raft_consensus.cc:2781] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:52:25.821772 13175 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:25.822165 13175 raft_consensus.cc:3037] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:25.822772 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:25.823635 13255 raft_consensus.cc:2726] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:25.797470 13187 raft_consensus.cc:2781] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:25.824409 13187 raft_consensus.cc:686] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:25.825263 13187 consensus_queue.cc:237] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:25.823218 11604 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:25.826627 13196 raft_consensus.cc:979] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:25.827167 13196 raft_consensus.cc:1067] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:25.827304 13246 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:25.828079 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:25.830192 10837 raft_consensus.cc:2454] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:52:25.830298 10891 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:25.831393 10891 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:25.832054 13200 raft_consensus.cc:2781] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:25.832429 13200 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:25.833006 13200 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:25.833052 13178 raft_consensus.cc:2726] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:25.832376 10891 leader_election.cc:302] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:25.836081 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:25.838091 11604 leader_election.cc:302] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:25.838619 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:25.840795 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.841836 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:25.842374 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.843492 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:25.845184 10891 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:25.846063 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:25.790805 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:25.846271 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:25.850718 11635 raft_consensus.cc:1226] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:25.847796 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:25.849484 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:25.847682 10891 leader_election.cc:302] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:25.851436 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.852445 11604 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:25.854305 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:25.856920 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:25.860373 11604 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:25.860257 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:25.856096 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:25.864486 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:25.865314 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:25.866679 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:25.864158 10891 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:25.869973 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:25.871331 11606 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:25.846415 13167 raft_consensus.cc:2781] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:25.873767 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:25.846674 13180 raft_consensus.cc:2726] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:25.874027 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:25.874758 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:25.846962 13196 raft_consensus.cc:2726] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:25.850510 13255 raft_consensus.cc:2726] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:25.878255 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:25.878793 11017 raft_consensus.cc:1226] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:25.880223 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:25.881283 13184 raft_consensus.cc:2726] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 15:52:25.869871 13099 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:25.882496 13177 raft_consensus.cc:2781] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:52:25.882992 13180 raft_consensus.cc:2726] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 15:52:25.858671 10838 raft_consensus.cc:3037] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:25.858844 13168 raft_consensus.cc:2726] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:25.870221 13178 raft_consensus.cc:2781] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:25.939621 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:52:25.936856 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 81535.9us
W20260809 15:52:25.959184 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 87925.9us
I20260809 15:52:25.878995 13167 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:25.959334 10847 raft_consensus.cc:1226] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 3. Current term is 4. Ops: []
W20260809 15:52:25.937407 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 73318.8us
I20260809 15:52:25.962178 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:25.966572 10843 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:25.968698 10891 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:25.972002 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 95243.6us
I20260809 15:52:25.960222 11606 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:25.974526 13229 raft_consensus.cc:2781] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:52:25.933179 13177 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:25.976981 11606 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:25.978168 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:25.939893 13178 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:25.975279 13229 raft_consensus.cc:686] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:25.967643 13167 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:26.057816 10891 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:26.049351 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 65130.4us
I20260809 15:52:26.049793 13229 consensus_queue.cc:237] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:26.050024 13178 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:26.063414 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:26.071106 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:26.073069 11635 raft_consensus.cc:1261] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260809 15:52:26.073670 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:26.083842 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:26.070920 13184 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:26.074563 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:26.077752 10838 raft_consensus.cc:2454] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:26.091660 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.092335 11012 raft_consensus.cc:2454] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:26.095896 10844 raft_consensus.cc:1226] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:25.983361 13177 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:26.052902 13168 raft_consensus.cc:2726] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 15:52:25.963908 13254 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:26.146140 11635 raft_consensus.cc:1226] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:26.146579 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:26.147722 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
W20260809 15:52:26.148489 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 51130us
I20260809 15:52:26.150285 11606 leader_election.cc:302] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:26.149207 10891 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:26.152508 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 292174us
I20260809 15:52:26.062182 13232 raft_consensus.cc:2726] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:26.153429 11604 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:26.169557 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.170014 11604 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:26.170811 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:26.171574 11012 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50542
I20260809 15:52:26.081511 13207 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:26.175215 10926 raft_consensus.cc:2379] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 3: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:26.175117 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:26.176307 10929 raft_consensus.cc:2454] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:26.177434 10924 raft_consensus.cc:2379] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 3: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:26.181238 10978 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:26.177390 10811 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:26.149799 13235 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:26.186547 10978 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:26.188598 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.170471 13111 raft_consensus.cc:2781] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:26.172399 13165 raft_consensus.cc:482] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:26.177542 13269 raft_consensus.cc:2726] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:26.209071 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:26.186257 13227 raft_consensus.cc:2726] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:26.181964 13251 raft_consensus.cc:2726] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:26.210143 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:26.216883 13231 raft_consensus.cc:2726] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:26.216305 10920 raft_consensus.cc:1261] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:26.217442 11633 raft_consensus.cc:1226] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:26.218113 10927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52758 (request call id 569) took 1273 ms. Trace:
I20260809 15:52:26.184932 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:26.189850 13252 raft_consensus.cc:2726] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260809 15:52:26.222612 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 73576.8us
I20260809 15:52:26.279017 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.201439 13258 raft_consensus.cc:2726] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:26.282439 10927 rpcz_store.cc:276] 0809 15:52:24.944452 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:24.944553 (+ 101us) service_pool.cc:227] Handling call
0809 15:52:26.217918 (+1273365us) spinlock_profiling.cc:243] Waited 300 ms on lock 0x7b540030a2d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:26.218018 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":689390976}
I20260809 15:52:26.287681 11014 raft_consensus.cc:2454] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:26.293071 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:26.293664 10927 raft_consensus.cc:2379] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 4: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:26.180416 13207 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:26.293256 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:26.294453 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
W20260809 15:52:26.280062 13191 consensus_meta.cc:287] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.519s user 0.000s sys 0.012s
I20260809 15:52:26.296557 11640 raft_consensus.cc:2379] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 4: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:26.296828 11013 raft_consensus.cc:2454] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:26.297003 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.300137 11634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 528) took 1336 ms. Trace:
I20260809 15:52:26.301373 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:26.200312 13235 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:26.308914 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 122697us
I20260809 15:52:26.311782 10926 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:26.317716 10891 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:26.305354 10927 raft_consensus.cc:1226] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:26.215076 13165 raft_consensus.cc:504] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:26.319406 10891 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:26.311460 11634 rpcz_store.cc:276] 0809 15:52:24.964114 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:25.009321 (+ 45207us) service_pool.cc:227] Handling call
0809 15:52:25.220244 (+210923us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:25.759881 (+539637us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:26.299853 (+539972us) spinlock_profiling.cc:243] Waited 486 ms on lock 0x7b5400720bd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:26.300060 (+ 207us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1118343552}
I20260809 15:52:26.176013 13225 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.137s
I20260809 15:52:26.173560 13260 consensus_queue.cc:1046] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.143s
W20260809 15:52:26.324754 13185 consensus_meta.cc:287] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.585s user 0.009s sys 0.002s
I20260809 15:52:26.324975 13184 raft_consensus.cc:2726] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:26.346514 13255 raft_consensus.cc:2726] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:26.347194 13254 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:26.294049 13111 raft_consensus.cc:686] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:26.349825 11011 raft_consensus.cc:1261] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:26.350271 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.404700 11015 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:26.405679 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:26.349859 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.310395 13191 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:26.362712 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.394507 11012 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:26.406986 11012 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:26.395714 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.400660 10930 raft_consensus.cc:1226] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:26.419713 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:26.418255 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 231782us
I20260809 15:52:26.424193 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:26.424494 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 241284us
W20260809 15:52:26.425518 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 248177us
I20260809 15:52:26.426532 10811 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:26.426617 10978 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:26.427346 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.430485 10923 raft_consensus.cc:1226] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:26.402555 13270 raft_consensus.cc:2726] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:26.430229 10978 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:26.435304 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.433915 10811 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:26.436628 10978 leader_election.cc:302] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:26.437965 10978 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:26.443439 10978 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:26.445627 11606 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:26.445875 10811 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:26.446669 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:26.401266 13185 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:26.449180 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.433943 13218 raft_consensus.cc:2726] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:26.455106 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.439260 13258 raft_consensus.cc:2726] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:26.444518 13223 raft_consensus.cc:2726] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:26.442655 13252 raft_consensus.cc:2726] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:26.460067 10893 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:26.465787 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.466137 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:26.466905 10926 raft_consensus.cc:2379] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 4: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:26.468258 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:26.468988 10927 raft_consensus.cc:2379] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 2: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:26.451002 11632 raft_consensus.cc:1226] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:26.465920 10921 raft_consensus.cc:1226] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:26.479666 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:26.477648 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.486207 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.492818 11012 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:26.493759 11018 raft_consensus.cc:1226] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:26.455509 13199 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:26.460114 13267 raft_consensus.cc:2726] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:26.459883 13177 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:26.479441 13175 consensus_meta.cc:287] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.657s user 0.005s sys 0.005s
I20260809 15:52:26.509557 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:26.491117 13181 consensus_meta.cc:287] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.718s user 0.007s sys 0.009s
I20260809 15:52:26.479494 11640 raft_consensus.cc:1261] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 1 LEARNER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 1 index: 3. (index mismatch)
I20260809 15:52:26.506588 13284 raft_consensus.cc:2781] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:26.513547 13207 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:26.513322 13175 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:26.536244 13175 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:26.514493 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:26.541972 13200 consensus_meta.cc:287] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.708s user 0.007s sys 0.006s
I20260809 15:52:26.542372 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.514410 11633 raft_consensus.cc:1226] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:26.544337 13284 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:26.545141 13284 raft_consensus.cc:3037] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:26.542385 13200 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:26.514307 13177 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:26.547035 10922 raft_consensus.cc:1226] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:26.463119 13282 raft_consensus.cc:2781] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:26.546644 13287 consensus_queue.cc:1035] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.043s
I20260809 15:52:26.534013 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:26.549603 13165 leader_election.cc:288] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:26.484338 13254 raft_consensus.cc:2726] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:26.476392 13268 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:26.464962 13111 consensus_queue.cc:237] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:26.550627 13268 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:26.553344 13268 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:26.553779 13276 raft_consensus.cc:2781] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:26.554271 10931 raft_consensus.cc:1226] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:26.516953 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:26.519069 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.503949 13199 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:26.510807 13185 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:26.520310 13285 raft_consensus.cc:2726] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:26.520623 13178 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:26.525274 13191 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:26.541145 13235 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:26.556138 11014 raft_consensus.cc:2454] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:26.519773 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:26.562880 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 53623.6us
I20260809 15:52:26.562995 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.568130 13282 raft_consensus.cc:686] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:26.573246 11015 raft_consensus.cc:2454] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:26.554347 13276 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:26.579195 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:26.500423 13269 consensus_queue.cc:1046] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.070s
I20260809 15:52:26.590605 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:26.591926 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.592060 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:26.593170 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:26.596663 11012 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:26.582067 10811 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:26.582963 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:26.584041 11637 raft_consensus.cc:1226] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:26.597368 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:26.581708 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:26.529767 13181 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:26.606190 10893 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:26.616006 10891 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:26.581316 13200 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:26.623849 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.627456 13289 consensus_queue.cc:1046] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.007s
I20260809 15:52:26.628926 13178 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:26.629114 13225 consensus_queue.cc:1035] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:52:26.630839 13246 raft_consensus.cc:3032] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:26.631461 13246 raft_consensus.cc:729] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:26.633965 13270 raft_consensus.cc:2726] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:26.634560 10838 raft_consensus.cc:1261] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260809 15:52:26.580751 13276 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:26.617835 13259 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:26.641716 10842 raft_consensus.cc:1226] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:26.643353 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:26.571163 13167 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:26.647348 10893 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:26.655465 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:26.659327 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.659174 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:26.656174 10893 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:26.661031 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.664028 11641 raft_consensus.cc:1261] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 3 LEARNER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:52:26.674461 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.675568 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:26.677496 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:26.678573 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.678949 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:26.680298 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:26.675132 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:26.685951 13191 raft_consensus.cc:2726] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:26.687201 11635 raft_consensus.cc:1226] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:26.700623 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:26.690527 11012 raft_consensus.cc:3032] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:26.691311 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:26.705101 11012 raft_consensus.cc:729] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:26.633008 13246 consensus_queue.cc:260] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:26.616861 13282 consensus_queue.cc:237] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:26.706037 10926 raft_consensus.cc:1226] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:26.653523 13254 raft_consensus.cc:2726] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:26.658409 13162 raft_consensus.cc:2726] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 15:52:26.721650 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:26.723698 11604 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:26.726025 13290 raft_consensus.cc:2726] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 15:52:26.727690 11604 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:26.728704 13290 raft_consensus.cc:2726] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:26.730546 13167 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:26.786221 13250 log.cc:924] Time spent T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.840s user 0.004s sys 0.002s
W20260809 15:52:26.787657 13273 log.cc:924] Time spent T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.105s user 0.000s sys 0.000s
I20260809 15:52:26.731945 11604 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:26.710359 11633 raft_consensus.cc:1226] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:26.710577 13269 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:26.827167 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
W20260809 15:52:26.828284 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 96657.9us
I20260809 15:52:26.716482 11012 consensus_queue.cc:260] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:26.829090 11604 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:26.829650 11012 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:26.679296 13178 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
W20260809 15:52:26.801453 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 86054.7us
I20260809 15:52:26.802714 10922 raft_consensus.cc:1261] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260809 15:52:26.720765 13225 consensus_queue.cc:1046] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.112s
W20260809 15:52:26.890887 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 61871.6us
I20260809 15:52:26.895136 11012 raft_consensus.cc:1261] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260809 15:52:26.899674 11604 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:26.900619 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 447171us
I20260809 15:52:26.790382 13131 consensus_queue.cc:1046] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.076s
I20260809 15:52:26.904138 13261 consensus_queue.cc:1046] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.214s
I20260809 15:52:26.905323 13261 raft_consensus.cc:3032] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:26.905506 13181 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:26.905721 13261 raft_consensus.cc:729] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:26.906143 13181 raft_consensus.cc:3032] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:26.907745 10978 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:26.794139 13191 consensus_queue.cc:1046] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260809 15:52:26.890868 13199 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:26.896222 13174 raft_consensus.cc:2726] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 15:52:26.892762 13187 raft_consensus.cc:2726] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 15:52:26.915531 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:26.921099 11633 raft_consensus.cc:2379] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 4: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
W20260809 15:52:26.923233 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 185452us
I20260809 15:52:26.924664 13259 raft_consensus.cc:2726] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:26.925700 13246 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:26.926518 10978 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:26.837415 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:26.928577 10737 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33062 (request call id 392) took 1111 ms. Trace:
I20260809 15:52:26.926693 11604 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:26.936825 11604 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:26.935799 10737 rpcz_store.cc:276] 0809 15:52:25.816744 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:25.820353 (+ 3609us) service_pool.cc:227] Handling call
0809 15:52:26.928508 (+1108155us) inbound_call.cc:173] Queueing success response
Metrics: {"reported_tablets":1}
I20260809 15:52:26.938555 11604 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:26.940083 10978 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:26.941949 10809 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:26.944403 10809 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:26.826984 13269 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:26.947837 10809 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:26.948671 10978 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:26.911926 13165 raft_consensus.cc:2781] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:26.912348 13270 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:26.910584 13227 raft_consensus.cc:2726] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:26.960657 13280 raft_consensus.cc:2781] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:26.958230 13128 raft_consensus.cc:2726] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:26.911034 13181 raft_consensus.cc:729] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:27.019080 13247 raft_consensus.cc:3032] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:27.019609 13247 raft_consensus.cc:729] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:27.019152 13181 consensus_queue.cc:260] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:27.020968 13181 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:27.001319 13296 raft_consensus.cc:2726] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:27.023255 13280 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:27.023732 13280 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:26.913609 13207 consensus_queue.cc:1046] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.053s
I20260809 15:52:26.956580 13258 raft_consensus.cc:2726] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:26.947325 13174 raft_consensus.cc:2726] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:26.967374 10978 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:27.001381 13259 raft_consensus.cc:2726] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:26.955461 13180 raft_consensus.cc:2726] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 15:52:26.947238 13175 raft_consensus.cc:2726] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
W20260809 15:52:26.998407 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 50651.5us
I20260809 15:52:27.041807 10809 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:26.913834 13217 consensus_queue.cc:1046] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.055s
I20260809 15:52:27.001273 10838 raft_consensus.cc:1261] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:26.965633 13216 raft_consensus.cc:2726] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:26.998623 10736 catalog_manager.cc:5495] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 3 to 4, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 4 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:26.967386 13261 consensus_queue.cc:260] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:27.029984 13260 raft_consensus.cc:3032] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Stepping down as leader of term 4
W20260809 15:52:27.000167 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 51039.5us
I20260809 15:52:26.967658 13165 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:27.003301 13284 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:26.915673 13235 consensus_queue.cc:1046] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
I20260809 15:52:26.946641 13225 consensus_queue.cc:1035] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
W20260809 15:52:26.988596 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 283167us
I20260809 15:52:27.037415 13272 raft_consensus.cc:3032] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Stepping down as leader of term 2
W20260809 15:52:27.028970 13288 log.cc:924] Time spent T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.112s user 0.000s sys 0.000s
I20260809 15:52:27.068835 13128 consensus_queue.cc:1046] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:27.072894 13297 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260809 15:52:27.026925 13270 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:27.054701 11604 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:27.020553 13247 consensus_queue.cc:260] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:27.088410 10926 raft_consensus.cc:1226] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:27.087350 10843 raft_consensus.cc:1226] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:27.089834 13269 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:27.088428 13247 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
W20260809 15:52:27.059383 13231 consensus_queue.cc:1159] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" responder_term: 3 status { last_received { term: 3 index: 3 } last_committed_idx: 3 last_received_current_leader { term: 3 index: 3 } }
I20260809 15:52:27.095147 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:27.095453 13284 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:27.068080 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:27.069610 10923 raft_consensus.cc:1226] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:27.037575 13258 raft_consensus.cc:2726] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 15:52:27.043484 10809 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:27.098165 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:27.098791 10931 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
W20260809 15:52:27.098961 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 55519.4us
I20260809 15:52:27.065659 13165 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:27.052325 13260 raft_consensus.cc:729] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:27.070284 13272 raft_consensus.cc:729] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:27.081779 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:27.077782 13249 consensus_queue.cc:1046] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.004s
I20260809 15:52:27.045609 13259 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:27.087353 11011 raft_consensus.cc:1226] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:27.099815 10809 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:27.118369 10809 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:27.120261 10809 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:27.121513 10809 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:27.073323 13234 consensus_queue.cc:1046] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
W20260809 15:52:27.100695 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 46096.8us
W20260809 15:52:27.124089 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 156771us
W20260809 15:52:27.096907 13225 consensus_queue.cc:1159] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "c7aa762318cb4e97875333f10679a9f9" responder_term: 3 status { last_received { term: 3 index: 4 } last_committed_idx: 4 error { code: PRECEDING_ENTRY_DIDNT_MATCH status { code: ILLEGAL_STATE message: "Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)" } } last_received_current_leader { term: 3 index: 4 } }
W20260809 15:52:27.096695 13295 log.cc:924] Time spent T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.091s user 0.001s sys 0.000s
I20260809 15:52:27.115046 13249 raft_consensus.cc:3032] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:27.105297 13260 consensus_queue.cc:260] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:27.101532 13269 raft_consensus.cc:2726] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 15:52:27.117918 13256 consensus_queue.cc:1046] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.029s
I20260809 15:52:27.147868 13197 raft_consensus.cc:3032] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:27.147763 13272 consensus_queue.cc:260] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:27.102847 13287 raft_consensus.cc:2726] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:27.148665 13197 raft_consensus.cc:729] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:27.128536 10809 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:27.126096 13177 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260809 15:52:27.161962 10978 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
W20260809 15:52:27.162904 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 90573.7us
I20260809 15:52:27.163118 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:27.132017 13267 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:27.101665 13276 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:27.172349 10931 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:27.173027 10931 raft_consensus.cc:595] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:52:27.169546 11013 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:27.171232 10842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 524) took 1386 ms. Trace:
I20260809 15:52:27.173794 10842 rpcz_store.cc:276] 0809 15:52:25.780962 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:25.782657 (+ 1695us) service_pool.cc:227] Handling call
0809 15:52:25.782916 (+ 259us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:26.644443 (+861527us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:27.167450 (+523007us) spinlock_profiling.cc:243] Waited 859 ms on lock 0x7b54003aeb54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:27.167575 (+ 125us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1974671744}
I20260809 15:52:27.150025 13272 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:27.145762 13249 raft_consensus.cc:729] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:27.180212 10923 raft_consensus.cc:1226] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:27.180665 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:27.162575 13297 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:27.187218 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:27.205207 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:27.165482 13260 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:27.206507 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:27.208168 10924 raft_consensus.cc:1226] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:27.191809 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:27.188419 10978 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:27.211158 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:27.196537 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:27.211763 10978 leader_election.cc:302] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:27.176749 13179 raft_consensus.cc:2726] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:27.212853 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:27.216267 13217 raft_consensus.cc:2726] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:27.218529 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260809 15:52:27.172096 13287 raft_consensus.cc:2726] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260809 15:52:27.197520 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 69046.6us
I20260809 15:52:27.117990 13175 consensus_queue.cc:1035] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
W20260809 15:52:27.151379 13180 consensus_queue.cc:1159] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "20730d75ad954a82b126148f8152a9f3" responder_term: 3 status { last_received { term: 2 index: 5 } last_committed_idx: 5 error { code: INVALID_TERM status { code: ILLEGAL_STATE message: "Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []" } } last_received_current_leader { term: 0 index: 0 } }
I20260809 15:52:27.202993 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:27.240942 11013 raft_consensus.cc:3032] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:27.241384 11013 raft_consensus.cc:729] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:27.242286 11013 consensus_queue.cc:260] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:27.242676 13270 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:27.246308 13249 consensus_queue.cc:260] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:27.248970 13249 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:27.203655 11604 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
W20260809 15:52:27.252542 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 49044.5us
I20260809 15:52:27.211866 11637 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:27.148195 13284 consensus_queue.cc:1046] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.021s
I20260809 15:52:27.198958 11636 raft_consensus.cc:1226] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:27.220468 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:52:27.232950 10809 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:27.243698 11013 raft_consensus.cc:3037] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:27.263504 11016 raft_consensus.cc:3032] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:27.264233 11016 raft_consensus.cc:729] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:27.220335 13258 raft_consensus.cc:2726] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 15:52:27.269416 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 57740.8us
I20260809 15:52:27.270493 10978 leader_election.cc:302] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:27.209017 13300 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:27.271291 11016 consensus_queue.cc:260] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:27.276773 11016 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:27.274964 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:27.273584 10809 leader_election.cc:302] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:27.280333 10978 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:27.280704 11637 raft_consensus.cc:2454] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:27.281714 10978 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:27.220850 13128 raft_consensus.cc:2726] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:27.278878 10809 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:27.283881 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 105566us
I20260809 15:52:27.286042 10811 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:27.290318 11637 raft_consensus.cc:1226] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:27.294190 10978 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:27.300261 11013 raft_consensus.cc:1261] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260809 15:52:27.301429 13308 raft_consensus.cc:2726] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:27.303097 13308 raft_consensus.cc:2726] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:27.304186 13308 raft_consensus.cc:2726] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:27.307961 10978 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:27.205821 13105 consensus_queue.cc:237] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 2, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:27.311779 10924 raft_consensus.cc:1226] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:27.287422 13162 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:27.282948 13179 raft_consensus.cc:2726] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:27.210047 13230 consensus_queue.cc:1046] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.075s
I20260809 15:52:27.278903 13187 raft_consensus.cc:2781] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:27.340248 13187 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:27.308498 13314 raft_consensus.cc:2726] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:27.310956 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:27.342510 11636 raft_consensus.cc:2379] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 5: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:27.331296 11016 raft_consensus.cc:1261] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 2 index: 5. (index mismatch)
I20260809 15:52:27.315403 13276 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:27.287240 13235 raft_consensus.cc:2726] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:27.292678 13258 raft_consensus.cc:2726] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:27.353060 13296 raft_consensus.cc:3032] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:27.353416 13297 raft_consensus.cc:2726] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:27.353456 13296 raft_consensus.cc:729] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:27.331234 13284 raft_consensus.cc:2726] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:27.339548 13230 raft_consensus.cc:3032] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:27.299605 13217 raft_consensus.cc:2726] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:27.336042 13162 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:27.229784 13197 consensus_queue.cc:260] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:27.340997 13187 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:27.276636 13306 consensus_queue.cc:1046] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.078s
I20260809 15:52:27.335328 13280 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:27.277642 13303 consensus_queue.cc:1046] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.091s
I20260809 15:52:27.390187 10838 raft_consensus.cc:1261] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260809 15:52:27.390772 13306 raft_consensus.cc:3032] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:27.395357 13306 raft_consensus.cc:729] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:27.367118 13230 raft_consensus.cc:729] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:27.401409 13303 raft_consensus.cc:3032] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:27.402117 13303 raft_consensus.cc:729] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:27.401813 13230 consensus_queue.cc:260] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:27.327778 13229 consensus_queue.cc:1035] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.035s
I20260809 15:52:27.396706 13306 consensus_queue.cc:260] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:27.390240 13305 log.cc:924] Time spent T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.141s user 0.000s sys 0.007s
I20260809 15:52:27.423305 13251 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:27.419921 13187 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:27.374751 13227 consensus_queue.cc:1035] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
I20260809 15:52:27.404523 13230 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:27.424110 13251 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:27.409144 13296 consensus_queue.cc:260] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:27.539160 13304 raft_consensus.cc:3032] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Stepping down as leader of term 2
W20260809 15:52:27.592602 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 313771us
I20260809 15:52:27.639945 10809 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:27.641396 10809 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:27.625804 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:27.403741 13303 consensus_queue.cc:260] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:27.648941 10809 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:27.652447 10809 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:27.658998 10809 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:27.660192 13231 raft_consensus.cc:2781] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:27.649531 13297 raft_consensus.cc:2726] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:27.660904 13231 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:27.661451 13231 raft_consensus.cc:3037] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:27.636317 13306 raft_consensus.cc:3037] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:27.664361 13296 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:27.664580 10809 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:27.674232 10809 leader_election.cc:302] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:27.677491 11012 raft_consensus.cc:1226] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:27.680961 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:27.652447 13304 raft_consensus.cc:729] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
W20260809 15:52:27.670209 13165 consensus_meta.cc:287] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.566s user 0.013s sys 0.001s
I20260809 15:52:27.702402 13165 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:27.704072 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:27.704970 13165 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:27.704867 10837 raft_consensus.cc:2454] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:27.689522 13179 raft_consensus.cc:2726] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:27.701016 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:27.670542 13297 raft_consensus.cc:2726] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:27.704169 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:27.698442 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:27.707271 11010 raft_consensus.cc:1226] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:27.697100 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:27.713141 10841 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:27.690145 13162 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:27.689774 13276 raft_consensus.cc:2726] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:27.727437 10844 raft_consensus.cc:1226] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:27.665208 13260 raft_consensus.cc:2781] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260809 15:52:27.709983 13259 consensus_meta.cc:287] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.577s user 0.009s sys 0.005s
I20260809 15:52:27.716866 13187 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:27.747985 13251 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:27.735119 13260 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:27.739395 13259 raft_consensus.cc:2726] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:27.739086 13280 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:27.775509 10843 raft_consensus.cc:1226] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:27.783062 13207 raft_consensus.cc:3032] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:27.783697 13207 raft_consensus.cc:729] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:27.788468 11632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 544) took 1138 ms. Trace:
W20260809 15:52:27.758612 13272 consensus_meta.cc:287] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.565s user 0.005s sys 0.012s
I20260809 15:52:27.813179 13261 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:27.830829 13310 raft_consensus.cc:3032] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:52:27.831642 13310 raft_consensus.cc:729] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:27.839047 13289 raft_consensus.cc:3032] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:27.839614 13289 raft_consensus.cc:729] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:27.773202 13260 raft_consensus.cc:3037] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:27.846930 11011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2056) took 1302 ms. Trace:
I20260809 15:52:27.847425 11011 rpcz_store.cc:276] 0809 15:52:26.544093 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:26.546246 (+ 2153us) service_pool.cc:227] Handling call
0809 15:52:26.546585 (+ 339us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:27.116395 (+569810us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:27.846583 (+730188us) spinlock_profiling.cc:243] Waited 540 ms on lock 0x7b540040b354. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:27.846810 (+ 227us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1243142912}
I20260809 15:52:27.849642 13282 raft_consensus.cc:3032] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:27.850291 13282 raft_consensus.cc:729] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:27.851809 13282 consensus_queue.cc:260] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:27.879748 10978 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.571s user 0.021s sys 0.024s
W20260809 15:52:27.880138 10978 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.571s user 0.021s sys 0.024s
W20260809 15:52:27.880702 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 572826us
I20260809 15:52:27.881462 10978 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
W20260809 15:52:27.886502 11606 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.557s user 0.003s sys 0.051s
I20260809 15:52:27.854071 13282 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
W20260809 15:52:27.889709 11606 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.560s user 0.003s sys 0.051s
W20260809 15:52:27.892954 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 563881us
I20260809 15:52:27.900262 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:27.902556 13165 raft_consensus.cc:2726] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:27.902695 10978 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:27.903908 10841 raft_consensus.cc:1226] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:27.904904 13165 raft_consensus.cc:2726] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:27.903851 10978 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:27.914469 10978 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:27.917387 10978 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:27.785231 13207 consensus_queue.cc:260] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:27.916728 13235 raft_consensus.cc:2726] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:27.833091 13310 consensus_queue.cc:260] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:27.943507 13310 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:27.774694 13304 consensus_queue.cc:260] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:27.953094 13276 consensus_queue.cc:1159] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" responder_term: 2 status { last_received { term: 2 index: 2 } last_committed_idx: 2 last_received_current_leader { term: 2 index: 2 } }
I20260809 15:52:27.921427 13284 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:27.955029 13284 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:27.956674 13197 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:27.922161 13296 raft_consensus.cc:2726] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260809 15:52:27.970436 13300 consensus_meta.cc:287] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.677s user 0.007s sys 0.009s
I20260809 15:52:27.970939 13300 raft_consensus.cc:2726] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:27.971560 13175 consensus_queue.cc:1035] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260809 15:52:27.965894 13231 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:27.978032 10847 raft_consensus.cc:1226] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:27.980300 13299 consensus_queue.cc:1046] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 15:52:27.967932 13207 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:27.789153 11632 rpcz_store.cc:276] 0809 15:52:26.650197 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:26.653271 (+ 3074us) service_pool.cc:227] Handling call
0809 15:52:26.787251 (+133980us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:27.010179 (+222928us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:27.010216 (+ 37us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:27.086390 (+ 76174us) log.cc:842] Serialized 480 byte log entry
0809 15:52:27.192328 (+105938us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:27.208674 (+ 16346us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:27.217948 (+ 9274us) spinlock_profiling.cc:243] Waited 2.11 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:52:27.787919 (+569971us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b5400533754. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:27.788008 (+ 89us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:27.788092 (+ 84us) raft_consensus.cc:1671] finished
0809 15:52:27.788134 (+ 42us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:27.788376 (+ 242us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":486458112,"thread_start_us":66813,"threads_started":1,"wal-append.queue_time_us":101779}
W20260809 15:52:27.992074 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 74774.4us
I20260809 15:52:27.993376 13300 consensus_queue.cc:1046] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:27.980858 13231 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:27.994710 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:27.976874 13284 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:27.996819 13218 raft_consensus.cc:2726] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:27.991390 13111 consensus_queue.cc:1046] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.012s
I20260809 15:52:28.004242 13303 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:27.924144 13289 consensus_queue.cc:260] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:28.016538 11603 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.604s user 0.002s sys 0.051s
W20260809 15:52:28.034981 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 385569us
W20260809 15:52:28.044651 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 50622.4us
W20260809 15:52:28.025888 13249 consensus_meta.cc:287] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.776s user 0.014s sys 0.005s
I20260809 15:52:28.025712 13289 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:28.087700 13196 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:52:28.079442 13230 consensus_meta.cc:287] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.516s user 0.009s sys 0.003s
I20260809 15:52:28.093341 13304 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:28.102386 13296 raft_consensus.cc:3032] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
W20260809 15:52:28.146363 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 73890.1us
W20260809 15:52:28.087395 11603 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.712s user 0.002s sys 0.051s
I20260809 15:52:28.104833 13296 raft_consensus.cc:729] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
W20260809 15:52:28.151093 10811 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.864s user 0.004s sys 0.037s
W20260809 15:52:28.154646 13267 consensus_meta.cc:287] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.974s user 0.006s sys 0.012s
W20260809 15:52:28.162562 10811 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.876s user 0.004s sys 0.037s
I20260809 15:52:28.157812 13256 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:28.095306 13175 raft_consensus.cc:3032] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:28.163388 10978 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:28.165795 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 879748us
I20260809 15:52:28.092216 13167 consensus_queue.cc:1046] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260809 15:52:28.158962 13230 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:52:28.166221 13216 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher.
I20260809 15:52:28.166008 13218 consensus_queue.cc:1046] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.004s
I20260809 15:52:28.113111 13241 raft_consensus.cc:3032] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 3 LEADER]: Stepping down as leader of term 3
W20260809 15:52:28.171813 11605 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.812s user 0.005s sys 0.016s
W20260809 15:52:28.167416 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 75241us
W20260809 15:52:28.178437 11605 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.819s user 0.005s sys 0.016s
W20260809 15:52:28.172921 10890 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.598s user 0.012s sys 0.025s
W20260809 15:52:28.179477 10890 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.605s user 0.012s sys 0.025s
I20260809 15:52:28.096100 13260 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.175613 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:52:28.182858 10893 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.548s user 0.009s sys 0.033s
W20260809 15:52:28.183631 10893 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.549s user 0.009s sys 0.033s
I20260809 15:52:28.188202 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:28.188735 10839 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:28.175469 13296 consensus_queue.cc:260] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:28.194607 11013 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.503s user 0.004s sys 0.046s
W20260809 15:52:28.194937 11013 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.503s user 0.004s sys 0.046s
I20260809 15:52:28.165587 13267 raft_consensus.cc:2726] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:28.091516 13168 consensus_queue.cc:1046] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.073s
I20260809 15:52:28.169488 13175 raft_consensus.cc:729] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:28.174432 13310 raft_consensus.cc:2781] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:28.216311 10840 raft_consensus.cc:1226] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:28.216976 10843 raft_consensus.cc:1226] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:28.221364 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:28.178380 13241 raft_consensus.cc:729] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:28.195300 13299 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:28.226403 13296 raft_consensus.cc:3037] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:28.228055 10924 raft_consensus.cc:1261] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260809 15:52:28.229498 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:28.174767 13249 consensus_queue.cc:1046] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260809 15:52:28.233502 10929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:54216 (request call id 1529) took 1766 ms. Trace:
I20260809 15:52:28.213035 13321 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:28.239821 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:28.169000 13256 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.093916 13196 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.242275 10929 rpcz_store.cc:276] 0809 15:52:26.459962 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:26.460188 (+ 226us) service_pool.cc:227] Handling call
0809 15:52:28.219497 (+1759309us) spinlock_profiling.cc:243] Waited 1.72 s on lock 0x7b540034f154. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478af0 00007f79694fbf2a 00007f796e438c6e 00007f7967b2f807 00007f7967b2f792 00007f7967b2f6c1 00007f7967b2f63c 00007f7967b2e8f2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:28.226171 (+ 6674us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3956126976}
I20260809 15:52:28.174193 13230 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.224115 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:28.220338 13232 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:28.262512 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:28.229100 13310 raft_consensus.cc:686] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 5 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:28.224315 13315 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
W20260809 15:52:28.260352 13311 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.687s user 0.012s sys 0.046s
I20260809 15:52:28.232077 13251 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:28.238600 13184 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:28.239194 13211 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:28.261835 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:28.379257 13253 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:28.379822 13253 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.381678 13253 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:28.383230 13256 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:28.384771 13196 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:28.385365 13196 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:28.385880 13196 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.387097 13200 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:28.387600 13200 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.389413 13200 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:28.309774 13232 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.372491 13184 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:28.391727 13232 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:28.392441 13184 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:28.269968 13175 consensus_queue.cc:260] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.376685 13211 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.394659 13274 raft_consensus.cc:3032] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:28.395077 13274 raft_consensus.cc:729] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:28.396198 13235 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:28.395884 13274 consensus_queue.cc:260] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:28.396693 13235 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:28.396888 13241 consensus_queue.cc:260] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.309521 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:28.398447 11633 raft_consensus.cc:2454] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:28.241392 13181 raft_consensus.cc:2726] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:28.235517 13322 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:28.261138 13312 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:28.370143 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 50092.9us
W20260809 15:52:28.334292 13169 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.680s user 0.013s sys 0.060s
I20260809 15:52:28.410689 10923 raft_consensus.cc:2454] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
W20260809 15:52:28.414695 11017 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.680s user 0.015s sys 0.117s
W20260809 15:52:28.432333 11017 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.703s user 0.016s sys 0.117s
I20260809 15:52:28.236794 13299 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.398204 13267 raft_consensus.cc:2932] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
W20260809 15:52:28.418455 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 197213us
I20260809 15:52:28.444070 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
W20260809 15:52:28.328750 13311 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.864s user 0.018s sys 0.049s
I20260809 15:52:28.453904 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:28.262784 13318 consensus_queue.cc:1035] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.168s
I20260809 15:52:28.243520 13272 consensus_queue.cc:1046] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.170s
I20260809 15:52:28.396890 13274 raft_consensus.cc:3037] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:28.456012 13272 raft_consensus.cc:3032] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:28.456424 13272 raft_consensus.cc:729] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:28.437625 11606 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:28.457264 13272 consensus_queue.cc:260] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.334266 13165 consensus_queue.cc:1046] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.083s
I20260809 15:52:28.266659 13321 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:28.462751 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:28.398489 13241 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:28.453322 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:28.454265 11015 raft_consensus.cc:1226] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:28.454807 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:28.330518 13258 consensus_queue.cc:1046] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.087s
I20260809 15:52:28.458822 10927 raft_consensus.cc:1226] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:28.396198 13211 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:28.469928 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:28.470065 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:28.493044 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:28.492187 11012 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
W20260809 15:52:28.530196 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 70184.7us
W20260809 15:52:28.450677 13169 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.778s user 0.013s sys 0.060s
I20260809 15:52:28.536820 10924 raft_consensus.cc:1226] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:28.542886 10891 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:28.542913 10843 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:28.544257 11606 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:28.417527 13251 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.462296 13105 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:28.494977 13260 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:28.556550 10927 raft_consensus.cc:1226] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:28.559108 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:28.534485 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:28.417683 13315 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.564319 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:28.546962 13318 raft_consensus.cc:2726] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:28.540292 13235 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:28.569587 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 235698us
I20260809 15:52:28.551296 13211 raft_consensus.cc:2781] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:28.545352 13319 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:28.432837 13322 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.549743 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:28.606539 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:28.567198 10921 raft_consensus.cc:2379] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 3: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:28.644970 10978 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:28.555001 13200 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
W20260809 15:52:28.649617 13328 log.cc:924] Time spent T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.109s user 0.000s sys 0.000s
W20260809 15:52:28.653282 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 191596us
I20260809 15:52:28.567183 13326 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:28.458751 13310 consensus_queue.cc:237] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.586143 13169 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:28.675184 10811 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:28.656139 11009 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:52:28.605389 13211 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:28.569236 13336 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:28.572141 13197 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:28.565594 13177 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:28.568516 13260 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:28.681208 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 137011us
I20260809 15:52:28.450157 13312 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:28.684041 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 116645us
I20260809 15:52:28.566673 13105 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.676687 10929 raft_consensus.cc:2932] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Committing config change with OpId 4.7: config changed from index 6 to 7, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:28.703094 10926 raft_consensus.cc:2932] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:28.615015 13230 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:28.653770 13232 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:28.688644 13324 log.cc:924] Time spent T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.069s user 0.000s sys 0.000s
I20260809 15:52:28.610486 13264 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:28.610765 13319 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:28.678923 13200 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.739506 13232 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.739912 13339 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:28.740474 13339 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:28.703841 13231 raft_consensus.cc:2726] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:28.738023 13306 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:28.700006 13211 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:28.720983 13260 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.734314 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:28.786162 10842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 554) took 1529 ms. Trace:
I20260809 15:52:28.787519 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:28.790678 10841 raft_consensus.cc:2454] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:28.699026 13196 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:28.682113 13235 consensus_queue.cc:1046] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
W20260809 15:52:28.725803 13207 consensus_meta.cc:287] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.736s user 0.008s sys 0.004s
I20260809 15:52:28.745478 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:28.806488 11009 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:28.783411 13282 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:28.790460 10842 rpcz_store.cc:276] 0809 15:52:27.257213 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:27.257339 (+ 126us) service_pool.cc:227] Handling call
0809 15:52:27.257668 (+ 329us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:27.271454 (+ 13786us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:27.271483 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:27.271997 (+ 514us) log.cc:842] Serialized 26 byte log entry
0809 15:52:27.771914 (+499917us) spinlock_profiling.cc:243] Waited 2.61 ms on lock 0x7b180023db20. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f79665ad345 00007f796f3e4a39 00007f79665ca8bb 00007f79665c8cbf 00007f79665cae22 00007f79691745bb 00007f796917a063 00007f796917a411 00007f79694c75cf
0809 15:52:28.235298 (+463384us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:52:28.235346 (+ 48us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:28.235971 (+ 625us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:28.742692 (+506721us) raft_consensus.cc:1671] finished
0809 15:52:28.742717 (+ 25us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:28.786037 (+ 43320us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":13372,"raft.queue_time_us":14906,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":174,"spinlock_wait_cycles":6001408,"thread_start_us":463189,"threads_started":1,"wal-append.queue_time_us":1163760}
I20260809 15:52:28.721385 13233 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:28.737110 13105 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:28.734858 13330 log.cc:924] Time spent T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.402s user 0.001s sys 0.000s
W20260809 15:52:28.737839 13329 log.cc:924] Time spent T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.065s user 0.000s sys 0.000s
I20260809 15:52:28.696854 13178 raft_consensus.cc:2781] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:28.729192 13312 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:28.714498 13336 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.811122 11013 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:28.733865 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:28.798995 13303 consensus_meta.cc:287] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.790s user 0.012s sys 0.000s
W20260809 15:52:28.836331 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 111190us
I20260809 15:52:28.735390 13321 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:28.799597 13168 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:28.788786 13334 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:28.740967 13264 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.689905 13326 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:28.715466 13197 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.822249 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
W20260809 15:52:28.823293 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 178384us
I20260809 15:52:28.700567 13169 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:28.803922 10838 raft_consensus.cc:1226] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:28.826426 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:28.826256 13178 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:28.827901 11606 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:28.843307 13231 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:28.833918 13232 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:28.841687 13312 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:28.834257 13315 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:28.804800 13299 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:28.835601 11017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 1904) took 1157 ms. Trace:
I20260809 15:52:28.803947 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:28.789929 13306 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.848480 10978 leader_election.cc:302] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:28.847194 13251 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:28.899583 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 51185.3us
I20260809 15:52:28.900334 10978 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:28.849854 11639 raft_consensus.cc:1226] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:28.714685 13177 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.823257 13342 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:28.833014 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:28.855816 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:28.858270 13178 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:28.884629 10893 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:28.818755 13282 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.845232 13196 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:28.849431 13200 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:28.901755 10978 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:28.896700 11017 rpcz_store.cc:276] 0809 15:52:27.678012 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:27.678157 (+ 145us) service_pool.cc:227] Handling call
0809 15:52:27.678640 (+ 483us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:27.692352 (+ 13712us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:27.692389 (+ 37us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:27.698183 (+ 5794us) log.cc:842] Serialized 540 byte log entry
0809 15:52:28.432819 (+734636us) raft_consensus.cc:1631] Marking committed up to 2
0809 15:52:28.432869 (+ 50us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:28.432954 (+ 85us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:28.835362 (+402408us) raft_consensus.cc:1671] finished
0809 15:52:28.835387 (+ 25us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:28.835512 (+ 125us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":702919,"threads_started":1,"wal-append.queue_time_us":852007}
I20260809 15:52:28.903626 10920 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:28.904232 13319 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:28.904984 13331 log.cc:924] Time spent T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.574s user 0.001s sys 0.001s
I20260809 15:52:28.937695 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:28.905699 13197 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:28.825870 13233 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.906339 13169 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:28.906982 13216 raft_consensus.cc:3032] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:28.850911 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:28.919870 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:28.951592 11015 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:28.922372 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:28.847528 13168 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:28.928880 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 44345.9us
I20260809 15:52:28.952687 11009 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:28.953126 10893 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:28.902597 13235 raft_consensus.cc:2726] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:28.902979 13258 raft_consensus.cc:2726] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:28.899256 13322 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:28.939635 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:28.961678 10929 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:28.849857 13334 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:28.898469 13312 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:28.954864 10891 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:28.936910 13321 raft_consensus.cc:2781] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:28.898197 13231 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:28.960098 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 132257us
I20260809 15:52:28.944053 13351 raft_consensus.cc:2781] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:28.926879 13196 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:28.950577 13296 consensus_meta.cc:287] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.708s user 0.005s sys 0.005s
W20260809 15:52:28.966441 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 64756.3us
I20260809 15:52:28.927680 13349 consensus_queue.cc:1046] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260809 15:52:28.951190 13216 raft_consensus.cc:729] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:28.921710 13342 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:28.972431 11010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 403) took 1107 ms. Trace:
I20260809 15:52:28.973037 13177 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:28.960896 11634 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 3 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:28.966543 13260 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:28.963786 13235 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:28.980597 10891 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:28.967444 13321 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:28.967448 13258 raft_consensus.cc:2726] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:28.980321 11010 rpcz_store.cc:276] 0809 15:52:27.865130 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:27.865288 (+ 158us) service_pool.cc:227] Handling call
0809 15:52:27.906148 (+ 40860us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:27.906541 (+ 393us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:27.906577 (+ 36us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:27.907629 (+ 1052us) log.cc:842] Serialized 26 byte log entry
0809 15:52:28.171563 (+263934us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:28.171599 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:28.171673 (+ 74us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:28.847407 (+675734us) spinlock_profiling.cc:243] Waited 36.6 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:52:28.970251 (+122844us) raft_consensus.cc:1671] finished
0809 15:52:28.970271 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:28.972319 (+ 2048us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1304,"raft.run_cpu_time_us":243,"raft.run_wall_time_us":7777,"spinlock_wait_cycles":84140288,"thread_start_us":241230,"threads_started":1,"wal-append.queue_time_us":310594}
I20260809 15:52:28.987372 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:28.987744 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:28.991037 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:28.992200 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:28.987427 10736 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59504 (request call id 265) took 2148 ms. Trace:
I20260809 15:52:28.993530 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:28.966209 13339 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:29.002691 11606 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:29.022269 11606 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:28.981608 10893 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:29.004508 13344 raft_consensus.cc:2781] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:28.969805 13351 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:29.013230 10811 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:29.005154 13318 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:29.009305 13296 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:28.982133 13274 raft_consensus.cc:2781] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260809 15:52:29.024365 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 44775.7us
I20260809 15:52:29.035997 10893 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:29.006585 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:29.005836 13256 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:29.012153 13321 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:29.019173 10927 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:29.008864 13177 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:29.010349 13261 consensus_queue.cc:1046] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.020s
I20260809 15:52:29.028081 13351 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:29.006283 13359 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:29.034108 13274 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:29.038790 13326 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.011610 13235 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:29.047058 10978 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:29.077152 10978 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:29.017092 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:29.026463 13344 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:29.057720 13200 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:29.065510 13361 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:29.034308 13260 raft_consensus.cc:2781] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:29.067059 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.042513 13339 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:29.048009 13306 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:29.034544 13318 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.018885 10736 rpcz_store.cc:276] 0809 15:52:26.838592 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:26.839352 (+ 760us) service_pool.cc:227] Handling call
0809 15:52:28.987368 (+2148016us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 15:52:27.198055 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 15:52:27.199004 (+ 949us) write_op.cc:432] Acquiring schema lock in shared mode
0809 15:52:27.199036 (+ 32us) write_op.cc:435] Acquired schema lock
0809 15:52:27.199075 (+ 39us) tablet.cc:660] Decoding operations
0809 15:52:27.199539 (+ 464us) write_op.cc:620] Acquiring the partition lock for write op
0809 15:52:27.199715 (+ 176us) write_op.cc:641] Partition lock acquired for write op
0809 15:52:27.199754 (+ 39us) tablet.cc:683] Acquiring locks for 1 operations
0809 15:52:27.200115 (+ 361us) tablet.cc:699] Row locks acquired
0809 15:52:27.200175 (+ 60us) write_op.cc:260] PREPARE: finished
0809 15:52:27.200373 (+ 198us) write_op.cc:270] Start()
0809 15:52:27.200545 (+ 172us) write_op.cc:276] Timestamp: P: 1786290747200319 usec, L: 0
0809 15:52:27.200572 (+ 27us) op_driver.cc:331] REPLICATION: starting
0809 15:52:27.201701 (+ 1129us) log.cc:842] Serialized 527 byte log entry
0809 15:52:28.698015 (+1496314us) op_driver.cc:444] REPLICATION: finished
0809 15:52:28.821613 (+123598us) write_op.cc:301] APPLY: starting
0809 15:52:28.821698 (+ 85us) tablet.cc:1365] starting BulkCheckPresence
0809 15:52:28.821799 (+ 101us) tablet.cc:1368] finished BulkCheckPresence
0809 15:52:28.883967 (+ 62168us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 15:52:28.916314 (+ 32347us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 15:52:28.939704 (+ 23390us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 15:52:28.948612 (+ 8908us) write_op.cc:312] APPLY: finished
0809 15:52:28.962246 (+ 13634us) log.cc:842] Serialized 23 byte log entry
0809 15:52:28.968023 (+ 5777us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:52:28.971325 (+ 3302us) write_op.cc:454] Released schema lock
0809 15:52:28.979206 (+ 7881us) write_op.cc:341] FINISH: Updating metrics
0809 15:52:29.009097 (+ 29891us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:52:29.013890 (+ 4793us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":26360,"reported_tablets":1,"child_traces":[["op",{"apply.queue_time_us":99820,"apply.run_cpu_time_us":2769,"apply.run_wall_time_us":198857,"num_ops":1,"prepare.queue_time_us":133721,"prepare.run_cpu_time_us":59091,"prepare.run_wall_time_us":93059,"raft.queue_time_us":1145256,"raft.run_cpu_time_us":1608,"raft.run_wall_time_us":208193,"replication_time_us":1483786,"thread_start_us":1192300,"threads_started":4,"wal-append.queue_time_us":258174}]]}
I20260809 15:52:29.072573 13274 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:29.069341 13319 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:29.055928 13282 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.041963 13296 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:29.048067 13256 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.093262 13344 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
W20260809 15:52:29.114444 10719 catalog_manager.cc:5025] ChangeConfig:REMOVE_PEER RPC for tablet 0b2b0df042e44725a7c91deae3df1956 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica d74944fa9fa14ec78b0640ea709ff3aa is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 7 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:29.070189 13264 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:29.093084 13349 raft_consensus.cc:2726] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:29.124076 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.108273 13233 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.121095 13337 raft_consensus.cc:2726] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:29.088719 13342 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.041327 13216 consensus_queue.cc:260] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.099040 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:29.127331 13334 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:29.052214 13336 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:29.102741 13260 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:29.051879 13177 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.118844 13282 raft_consensus.cc:2726] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 15:52:29.132278 13304 consensus_meta.cc:287] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 1.027s user 0.006s sys 0.010s
I20260809 15:52:29.109972 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:29.137683 13312 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.124938 11634 raft_consensus.cc:1261] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 3 LEARNER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:52:29.154592 13235 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:29.108188 13360 raft_consensus.cc:2726] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:29.142845 11018 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:29.115444 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.143373 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.149962 13260 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:29.111617 13168 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.114516 13200 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.115486 13319 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:29.109953 13339 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:29.151450 13196 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.132233 13318 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.072394 13359 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.168911 13356 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:29.163148 13354 raft_consensus.cc:2781] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:29.170639 13256 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.161211 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:29.198203 13162 raft_consensus.cc:979] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower a53441872c0c416b8e2f26f52399f536 to start an election
I20260809 15:52:29.176908 13167 raft_consensus.cc:3032] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:29.183583 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:29.188392 13175 raft_consensus.cc:3037] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:29.167447 11640 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:29.165895 13216 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
W20260809 15:52:29.215404 13289 consensus_meta.cc:287] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 1.113s user 0.014s sys 0.000s
I20260809 15:52:29.189106 13234 raft_consensus.cc:2932] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Committing config change with OpId 4.7: config changed from index 6 to 7, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
W20260809 15:52:29.203341 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 179719us
I20260809 15:52:29.226819 13276 raft_consensus.cc:3032] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:29.195436 13362 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:29.195384 13360 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:29.209257 11010 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:29.209406 13354 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:29.210194 13167 raft_consensus.cc:729] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:29.213987 13200 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:29.214502 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:29.216981 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.195022 13235 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:29.229152 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:29.217499 13211 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.231530 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:29.232283 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:29.273233 10931 raft_consensus.cc:2379] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 7: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:29.173659 13177 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.216610 11633 raft_consensus.cc:2454] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:52:29.278009 13359 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.233192 13289 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:29.278687 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:29.279414 13289 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:29.282426 13274 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.234678 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:29.235383 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:29.255095 13249 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:29.237694 10843 raft_consensus.cc:1226] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:29.238016 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:29.207234 13296 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:29.240381 13345 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:29.114153 13361 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.259964 11606 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
W20260809 15:52:29.307518 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 47658.6us
I20260809 15:52:29.308320 11606 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:29.219168 13349 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:29.309893 13349 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.227362 13276 raft_consensus.cc:729] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:29.314960 13276 consensus_queue.cc:260] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.317198 13276 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:29.268988 10840 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
W20260809 15:52:29.215517 10923 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 1915) took 2011 ms (client timeout 1786 ms). Trace:
W20260809 15:52:29.318773 10923 rpcz_store.cc:269] 0809 15:52:27.204341 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:27.204979 (+ 638us) service_pool.cc:227] Handling call
0809 15:52:29.215335 (+2010356us) spinlock_profiling.cc:243] Waited 1.19 s on lock 0x7b540024d6d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:29.215450 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2738043264}
I20260809 15:52:29.272429 10927 raft_consensus.cc:2454] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:29.264133 11632 raft_consensus.cc:2454] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:52:29.287609 10844 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:29.235721 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:29.289973 10921 raft_consensus.cc:2454] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:29.292251 11639 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:29.328823 13167 consensus_queue.cc:260] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:29.330183 10922 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:29.331053 13167 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:29.330210 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:29.260461 13312 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:29.263825 13354 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:29.262163 13322 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:29.328015 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:29.337005 11606 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:29.345208 13352 raft_consensus.cc:2781] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:29.307916 13249 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.303314 13359 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:29.352048 13274 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:29.327082 13282 raft_consensus.cc:2726] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:29.313622 13349 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:29.365949 11606 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:29.375356 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:29.274240 13235 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.266362 13362 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:29.369835 13363 log.cc:924] Time spent T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.054s user 0.001s sys 0.000s
I20260809 15:52:29.370762 13319 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.352833 13178 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.361130 13177 raft_consensus.cc:2781] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:29.343669 13312 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:29.376072 11639 raft_consensus.cc:2454] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:29.339447 13318 raft_consensus.cc:2781] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:29.357501 13345 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.384291 13235 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:29.359426 13289 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.385121 13211 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:29.391320 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:29.415921 11017 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 4
I20260809 15:52:29.392807 11606 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:29.396337 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.364331 11634 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:29.402300 13060 raft_consensus.cc:2781] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:29.361444 13322 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:29.404448 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:29.400442 13339 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:29.379351 13349 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:29.413791 11015 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:29.425346 13345 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.427989 13282 raft_consensus.cc:2781] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:29.400013 13177 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:29.412070 13249 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:29.411238 13241 consensus_meta.cc:287] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.916s user 0.008s sys 0.006s
W20260809 15:52:29.436484 13323 log.cc:924] Time spent T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 1.160s user 0.004s sys 0.002s
I20260809 15:52:29.431074 11639 raft_consensus.cc:2932] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:29.375015 13359 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.407343 13318 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:29.429826 13178 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:29.438237 11633 raft_consensus.cc:2454] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:29.303094 13360 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.442687 13177 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:29.457065 13359 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.442054 13282 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:29.439108 13349 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:29.456341 13361 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.466444 11636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 553) took 1228 ms. Trace:
I20260809 15:52:29.431185 13362 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:29.465684 13282 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:29.476437 11642 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:29.490584 13165 raft_consensus.cc:3032] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:29.659039 11640 raft_consensus.cc:1226] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:29.659569 13165 raft_consensus.cc:729] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:29.633076 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:29.664223 10839 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:29.637118 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:29.666612 10844 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:29.639030 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:29.643250 11016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 398) took 1979 ms. Trace:
I20260809 15:52:29.669114 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:29.667670 11016 rpcz_store.cc:276] 0809 15:52:27.663695 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:27.663874 (+ 179us) service_pool.cc:227] Handling call
0809 15:52:27.666382 (+ 2508us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:27.666718 (+ 336us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:27.666746 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:27.673164 (+ 6418us) log.cc:842] Serialized 26 byte log entry
0809 15:52:27.780990 (+107826us) spinlock_profiling.cc:243] Waited 8.41 ms on lock 0x7b48000213d8. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796e39a7f1 00007f796e3918e7 00007f796e39177a 00007f796e48595f 00007f796e485627 00007f796e499b2c 00007f796e499ad8 00007f796e499a58 00007f796e499a13 00007f796e498cbc 00007f796d41abe6
0809 15:52:28.165971 (+384981us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:28.166011 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:28.172108 (+ 6097us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:29.410642 (+1238534us) spinlock_profiling.cc:243] Waited 38.8 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:52:29.642396 (+231754us) raft_consensus.cc:1671] finished
0809 15:52:29.642445 (+ 49us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:29.642827 (+ 382us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":202,"raft.run_cpu_time_us":3829,"raft.run_wall_time_us":111432,"spinlock_wait_cycles":108543744,"thread_start_us":453489,"threads_started":1,"wal-append.queue_time_us":524857}
I20260809 15:52:29.472086 11636 rpcz_store.cc:276] 0809 15:52:28.237899 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:28.238411 (+ 512us) service_pool.cc:227] Handling call
0809 15:52:28.267842 (+ 29431us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:28.645781 (+377939us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:28.645954 (+ 173us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:28.646103 (+ 149us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:28.646140 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:29.466207 (+820067us) spinlock_profiling.cc:243] Waited 346 ms on lock 0x7b54007a1554. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:29.466253 (+ 46us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:29.466390 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":796239744}
I20260809 15:52:29.669955 13181 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:29.670995 13181 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:29.635443 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:29.673157 11018 raft_consensus.cc:1261] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:52:29.673738 10840 raft_consensus.cc:2454] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
W20260809 15:52:29.647416 10893 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.576s user 0.012s sys 0.010s
W20260809 15:52:29.690342 10893 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.619s user 0.012s sys 0.010s
W20260809 15:52:29.690874 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 619792us
I20260809 15:52:29.582574 13344 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:29.696309 13344 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.697103 13196 raft_consensus.cc:1067] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Signalling peer a53441872c0c416b8e2f26f52399f536 to start an election
W20260809 15:52:29.698395 13315 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.688s user 0.011s sys 0.022s
W20260809 15:52:29.698987 13315 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.717s user 0.012s sys 0.023s
I20260809 15:52:29.508869 13177 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.674741 13181 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.676576 10922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52758 (request call id 611) took 1249 ms. Trace:
I20260809 15:52:29.678130 10891 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:29.702704 10922 rpcz_store.cc:276] 0809 15:52:28.426961 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:28.431137 (+ 4176us) service_pool.cc:227] Handling call
0809 15:52:28.992019 (+560882us) spinlock_profiling.cc:243] Waited 6.52 ms on lock 0x7b58000d1498. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f7966593851 00007f7966593702 00007f796f483b09 00007f796e3bf730 00007f796e48579b 00007f796e485821 00007f796e4351b0 00007f796e434a94 00007f7967b1cc04 00007f7967b1cb92
0809 15:52:29.676098 (+684079us) spinlock_profiling.cc:243] Waited 302 ms on lock 0x7b540025d0d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:29.676341 (+ 243us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":709963392}
I20260809 15:52:29.503407 13349 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:29.703536 13344 raft_consensus.cc:2781] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:29.504243 13207 raft_consensus.cc:3032] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:29.704492 13344 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:29.704838 13207 raft_consensus.cc:729] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:29.703796 10930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1820) took 1028 ms. Trace:
I20260809 15:52:29.509742 13231 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:29.517460 13218 raft_consensus.cc:3032] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:29.706454 13218 raft_consensus.cc:729] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:29.705518 10930 rpcz_store.cc:276] 0809 15:52:28.675458 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:28.675903 (+ 445us) service_pool.cc:227] Handling call
0809 15:52:28.699170 (+ 23267us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:28.743978 (+ 44808us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:28.744021 (+ 43us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:28.744075 (+ 54us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:28.744122 (+ 47us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:29.703285 (+959163us) spinlock_profiling.cc:243] Waited 44.6 ms on lock 0x7b540025c1d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:29.703363 (+ 78us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:29.703594 (+ 231us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":102527872}
I20260809 15:52:29.705052 13177 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.661743 13165 consensus_queue.cc:260] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:29.437856 13260 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.708242 13218 consensus_queue.cc:260] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:29.709569 10840 raft_consensus.cc:1261] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260809 15:52:29.659050 11604 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.815s user 0.000s sys 0.021s
W20260809 15:52:29.710335 11604 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.866s user 0.001s sys 0.021s
I20260809 15:52:29.711498 13231 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:29.575534 13312 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:29.648578 11603 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.576s user 0.000s sys 0.012s
W20260809 15:52:29.714040 11603 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.662s user 0.000s sys 0.013s
I20260809 15:52:29.714881 13060 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:29.715420 13060 raft_consensus.cc:3037] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:29.716483 13207 consensus_queue.cc:260] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.448968 13318 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:29.478295 13322 leader_election.cc:288] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.683552 11635 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:29.684578 13249 raft_consensus.cc:482] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:52:29.684725 13232 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.618s user 0.022s sys 0.033s
I20260809 15:52:29.676282 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:29.703637 10891 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:29.705044 13344 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:29.511247 13272 raft_consensus.cc:3037] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:29.712549 13231 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:29.678966 10838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45578 (request call id 1905) took 1983 ms. Trace:
I20260809 15:52:29.516885 13321 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.800268 13321 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:29.453362 13351 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.494275 13356 raft_consensus.cc:2726] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:29.800465 10737 catalog_manager.cc:5495] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 1 to 2, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 2 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:29.573460 13360 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:29.746632 13349 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:29.797169 10838 rpcz_store.cc:276] 0809 15:52:27.695139 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:27.695548 (+ 409us) service_pool.cc:227] Handling call
0809 15:52:28.783273 (+1087725us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:28.852491 (+ 69218us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:29.678492 (+826001us) spinlock_profiling.cc:243] Waited 1.05 s on lock 0x7b540005aad0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:29.678729 (+ 237us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2403674880}
I20260809 15:52:29.802770 13360 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:29.802925 13351 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:29.804069 10929 raft_consensus.cc:2454] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:29.804700 10839 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:29.802946 13349 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.807951 13231 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.808840 13349 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:29.753832 13310 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.738s user 0.010s sys 0.036s
W20260809 15:52:29.766752 10926 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.795s user 0.001s sys 0.012s
W20260809 15:52:29.814707 10926 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.843s user 0.001s sys 0.012s
I20260809 15:52:29.771018 13234 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.ChangeConfig from 127.0.0.1:45572 (request call id 904) took 5029 ms. Trace:
I20260809 15:52:29.815547 13243 raft_consensus.cc:482] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:29.816308 13243 raft_consensus.cc:504] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.774755 11641 raft_consensus.cc:2454] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:29.818487 13361 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:29.775614 13381 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:29.819474 11017 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:29.819344 13361 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.820179 13381 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:29.815304 13234 rpcz_store.cc:276] 0809 15:52:24.741430 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:24.741613 (+ 183us) service_pool.cc:227] Handling call
0809 15:52:25.204814 (+463201us) spinlock_profiling.cc:243] Waited 1.97 ms on lock 0x7b58001f3db0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f796946ef63 00007f79694d2aba 00007f79694eea7a 00007f79694efada 00007f79694eec8b 00007f79694fa084 00007f79694f9743 00007f79694f1c1a 00007f796e4356d9 00007f7967b1f184
0809 15:52:25.229020 (+ 24206us) log.cc:842] Serialized 480 byte log entry
0809 15:52:26.527292 (+1298272us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:52:26.840292 (+313000us) spinlock_profiling.cc:243] Waited 679 ms on lock 0x7b58002027b0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478af0 00007f796946d834 00007f7969472771 00007f79694726f9 00007f7969472689 00007f7969472651 00007f796947181d 00007f796f82f204 00007f796f82f039 00007f79665cb6b6 00007f79665ceb11
0809 15:52:29.770624 (+2930332us) inbound_call.cc:173] Queueing success response
Metrics: {"dns_us":242,"mutex_wait_us":18302,"raft.queue_time_us":90944,"raft.run_cpu_time_us":9324,"raft.run_wall_time_us":1126507,"spinlock_wait_cycles":1597834112,"thread_start_us":557914,"threads_started":2,"wal-append.queue_time_us":458759,"wal-append.run_cpu_time_us":14141,"wal-append.run_wall_time_us":2223839}
I20260809 15:52:29.775686 13218 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:29.821935 10722 catalog_manager.cc:5008] ChangeConfig:REMOVE_PEER RPC for tablet 5e8d12b5ef9e415cbcb25e62ab228546 with cas_config_opid_index 6: ChangeConfig:REMOVE_PEER succeeded (attempt 1)
W20260809 15:52:29.779769 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 66608.6us
W20260809 15:52:29.824023 13276 consensus_meta.cc:287] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.506s user 0.010s sys 0.007s
I20260809 15:52:29.778573 13207 raft_consensus.cc:3037] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
W20260809 15:52:29.825222 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 133223us
I20260809 15:52:29.824630 13234 raft_consensus.cc:3032] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:29.829649 13234 raft_consensus.cc:729] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:29.830478 13361 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.830981 11606 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:29.830646 13234 consensus_queue.cc:260] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:29.831991 10811 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.737s user 0.016s sys 0.039s
I20260809 15:52:29.832115 13361 raft_consensus.cc:2781] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260809 15:52:29.832543 10811 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.783s user 0.016s sys 0.040s
I20260809 15:52:29.832760 11606 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:29.833837 13269 raft_consensus.cc:2781] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:29.834553 13269 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:29.834060 13249 raft_consensus.cc:504] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.834918 13269 raft_consensus.cc:3037] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:29.816407 13354 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:29.836417 13249 leader_election.cc:288] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:29.837311 13354 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.838394 13321 raft_consensus.cc:2781] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:29.838819 13321 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:29.839133 13321 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:29.842303 10924 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:29.843148 13282 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.844020 10926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1819) took 1168 ms. Trace:
I20260809 15:52:29.844395 10926 rpcz_store.cc:276] 0809 15:52:28.675162 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:28.675659 (+ 497us) service_pool.cc:227] Handling call
0809 15:52:28.702810 (+ 27151us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:28.703003 (+ 193us) raft_consensus.cc:1520] Early marking committed up to index 3
0809 15:52:28.971850 (+268847us) log.cc:842] Serialized 14 byte log entry
0809 15:52:29.843568 (+871718us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:29.843663 (+ 95us) raft_consensus.cc:1631] Marking committed up to 3
0809 15:52:29.843728 (+ 65us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:29.843822 (+ 94us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:29.843951 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":28236,"thread_start_us":843016,"threads_started":1,"wal-append.queue_time_us":795091}
W20260809 15:52:29.844657 10837 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.761s user 0.001s sys 0.015s
I20260809 15:52:29.830478 13381 leader_election.cc:288] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:29.846454 13234 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:29.848937 13260 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:29.774755 13165 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:29.803560 13360 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:29.766695 10809 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.956s user 0.011s sys 0.035s
I20260809 15:52:29.823906 11604 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:29.792681 13232 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.726s user 0.023s sys 0.033s
W20260809 15:52:29.849584 10837 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.818s user 0.002s sys 0.015s
I20260809 15:52:29.794237 13256 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:29.855518 13356 consensus_queue.cc:1035] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260809 15:52:29.855702 13256 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.858178 13256 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.848143 13269 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.859690 13256 raft_consensus.cc:2726] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 15:52:29.863730 10809 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.053s user 0.012s sys 0.035s
I20260809 15:52:29.863652 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.864437 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.864609 11010 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:29.865366 13282 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:29.852766 10841 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.726s user 0.001s sys 0.020s
W20260809 15:52:29.833828 10978 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.683s user 0.004s sys 0.006s
W20260809 15:52:29.813905 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 110493us
I20260809 15:52:29.865840 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:29.851389 13310 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.838s user 0.011s sys 0.036s
W20260809 15:52:29.868078 10978 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.768s user 0.005s sys 0.006s
W20260809 15:52:29.868253 11641 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41482 (request call id 555) took 1629 ms (client timeout 2065 ms). Trace:
W20260809 15:52:29.868603 10841 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.797s user 0.002s sys 0.020s
W20260809 15:52:29.869624 11641 rpcz_store.cc:269] 0809 15:52:28.239293 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:28.239549 (+ 256us) service_pool.cc:227] Handling call
0809 15:52:29.865794 (+1626245us) spinlock_profiling.cc:243] Waited 1.16 s on lock 0x7b5400631854. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:29.868165 (+ 2371us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2659827584}
I20260809 15:52:29.872615 10891 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:29.872768 11642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41482 (request call id 552) took 1635 ms (client timeout 2104 ms). Trace:
W20260809 15:52:29.873262 11642 rpcz_store.cc:269] 0809 15:52:28.237515 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:28.237654 (+ 139us) service_pool.cc:227] Handling call
0809 15:52:29.872345 (+1634691us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54005505d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:29.872560 (+ 215us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2718647680}
I20260809 15:52:29.873878 10891 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:29.883117 13249 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:29.886772 13354 raft_consensus.cc:2781] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260809 15:52:29.874413 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 780682us
I20260809 15:52:29.832790 13361 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:29.863657 13243 leader_election.cc:288] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.875128 10891 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:29.876225 13260 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:29.875828 13333 raft_consensus.cc:2781] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:29.912842 11017 raft_consensus.cc:2454] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:52:29.915148 10924 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:29.900082 13354 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:29.907370 11606 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:29.908910 13361 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:29.898808 13382 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
W20260809 15:52:29.921402 13384 log.cc:924] Time spent T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.085s user 0.001s sys 0.001s
I20260809 15:52:29.921548 13338 raft_consensus.cc:2726] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 5 LEADER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:29.916060 13333 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:29.926299 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.917600 13354 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
W20260809 15:52:29.928697 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 46841.9us
I20260809 15:52:29.915478 13381 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:52:29.929888 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 45888.1us
I20260809 15:52:29.930637 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.938540 13352 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:29.939105 13352 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:29.923421 13322 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:29.941221 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 66221.9us
I20260809 15:52:29.875451 13337 consensus_queue.cc:1035] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.050s
I20260809 15:52:29.944017 13336 raft_consensus.cc:2781] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:29.943882 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.945055 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.945477 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:29.946220 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.946514 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.947539 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.946516 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.928244 13333 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:29.917449 13249 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:29.956126 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:29.956832 11010 raft_consensus.cc:2454] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:52:29.953780 11011 raft_consensus.cc:2454] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:29.959214 11018 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:29.963335 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:29.963404 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:29.964056 10844 raft_consensus.cc:3037] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:29.939958 13310 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:29.940574 13269 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:29.969293 11016 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:29.968819 11012 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:29.971756 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:29.972421 10930 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:29.976053 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.975201 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.981961 10838 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:29.981873 13310 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.982388 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:29.983155 11011 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:29.977741 13243 raft_consensus.cc:2781] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:29.984171 13243 raft_consensus.cc:686] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:29.984311 13310 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:29.986204 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.990382 13276 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:29.990522 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.991050 13276 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.992602 10840 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:29.993111 13276 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:29.994620 11632 raft_consensus.cc:1226] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:29.996770 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:29.998221 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.977133 13322 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:29.999549 13282 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:30.000324 13322 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:30.000193 13282 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.002130 13249 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:30.002530 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:29.969076 13381 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.005296 13381 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:29.968976 13336 raft_consensus.cc:686] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:29.985263 13243 consensus_queue.cc:237] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.006421 11015 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:30.006712 13336 consensus_queue.cc:237] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.008659 11011 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:30.010211 13359 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:30.010833 13359 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.011904 13282 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:30.012588 10978 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.84.2:32783 timed out after 1.786s (SENT) [suppressed 27 similar messages]
I20260809 15:52:30.012918 13359 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:29.977594 13231 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:30.014446 13231 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:30.016135 10840 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:30.016462 13231 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:30.018390 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:30.019003 11016 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:30.020920 10838 raft_consensus.cc:2454] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:30.022233 10924 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:30.023849 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.023852 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.024545 10847 raft_consensus.cc:2454] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:52:30.024614 10843 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:30.025614 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.025743 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.026343 10843 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:30.026906 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.027357 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.027777 10847 raft_consensus.cc:2379] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 3: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:30.028486 10843 raft_consensus.cc:2396] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260809 15:52:30.028666 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.029311 10924 raft_consensus.cc:2454] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:29.950043 13260 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:30.029983 13289 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:30.030577 13369 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:30.031097 10837 raft_consensus.cc:2454] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:52:30.031174 13369 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.032235 13260 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:30.032771 13260 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:30.033119 11642 raft_consensus.cc:1226] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:30.033339 13369 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:30.030645 13289 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:30.034653 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:30.036278 13289 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:30.036798 13289 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:30.038882 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:30.040690 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.041114 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.041492 10922 raft_consensus.cc:2454] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:29.969314 13382 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.042272 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.042914 11013 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 1.
I20260809 15:52:30.044490 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.044507 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.045751 11606 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:30.047049 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.047747 13232 raft_consensus.cc:2781] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:30.048101 13382 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:30.048305 13232 raft_consensus.cc:686] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 3 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:30.033910 13369 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:30.049314 13369 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:30.049479 13232 consensus_queue.cc:237] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.051282 13369 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:30.037387 13289 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:30.029335 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.059077 11606 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:30.033917 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:30.017352 13344 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.061777 10844 raft_consensus.cc:2454] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:30.059017 10893 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:30.068696 10930 raft_consensus.cc:2454] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:30.067920 11606 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:30.062928 13233 raft_consensus.cc:2781] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:30.052313 13333 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:30.097608 13344 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:30.100162 13352 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.102820 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:30.033278 13260 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:30.107892 13334 raft_consensus.cc:2781] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:30.109023 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:30.109870 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:30.110800 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:30.112617 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:30.114434 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:30.115118 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:30.117621 13360 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:30.122148 13333 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:30.122900 13386 log.cc:924] Time spent T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.055s user 0.001s sys 0.000s
I20260809 15:52:30.122735 13321 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:30.088167 10811 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:30.130293 11016 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:30.132901 11012 raft_consensus.cc:2454] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:52:30.133066 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:30.138458 10926 raft_consensus.cc:1226] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:30.134246 11606 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:30.134402 11011 raft_consensus.cc:2454] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 1.
I20260809 15:52:30.140976 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:30.103813 13233 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:30.143442 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.147442 11606 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:30.149314 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.128341 13383 raft_consensus.cc:2781] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:30.129271 13334 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:30.153146 10843 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:30.156337 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:30.156627 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 144080us
I20260809 15:52:30.157460 10930 raft_consensus.cc:2379] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 5: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:30.160579 10841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 575) took 1124 ms. Trace:
I20260809 15:52:30.160598 10924 raft_consensus.cc:2379] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 4: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
W20260809 15:52:30.160588 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 95250.1us
I20260809 15:52:30.160378 11014 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:30.093039 13060 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.165784 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:30.173296 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:30.151376 13233 raft_consensus.cc:3037] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:30.146729 13289 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:30.152065 13359 raft_consensus.cc:2781] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:30.176928 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:30.183665 11011 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 3
I20260809 15:52:30.161209 11632 raft_consensus.cc:2379] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 3: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:30.177903 11606 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
W20260809 15:52:30.178256 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 219451us
I20260809 15:52:30.127642 13318 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.179299 11010 raft_consensus.cc:2454] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:30.161562 13334 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:30.153415 13383 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:30.187723 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:30.188648 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:30.190042 10840 raft_consensus.cc:2354] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4.
I20260809 15:52:30.192306 10811 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:30.166678 13234 raft_consensus.cc:2781] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:30.160946 10841 rpcz_store.cc:276] 0809 15:52:29.027096 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:29.029080 (+ 1984us) service_pool.cc:227] Handling call
0809 15:52:29.050866 (+ 21786us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:29.051101 (+ 235us) raft_consensus.cc:1520] Early marking committed up to index 7
0809 15:52:29.897783 (+846682us) log.cc:842] Serialized 14 byte log entry
0809 15:52:30.095837 (+198054us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:30.123839 (+ 28002us) raft_consensus.cc:1631] Marking committed up to 7
0809 15:52:30.128923 (+ 5084us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:30.133487 (+ 4564us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:30.151768 (+ 18281us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":782273,"prepare.run_cpu_time_us":56,"prepare.run_wall_time_us":61,"raft.queue_time_us":11313,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":157,"thread_start_us":962774,"threads_started":2}
I20260809 15:52:30.189455 11013 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 7.
I20260809 15:52:30.173416 13282 raft_consensus.cc:2781] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:30.210026 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:30.213418 11018 raft_consensus.cc:3037] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 3
I20260809 15:52:30.219399 13318 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:30.221163 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.221925 10842 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:30.226156 13356 consensus_queue.cc:1046] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 15:52:30.224639 10811 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:30.227047 13356 raft_consensus.cc:3032] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:30.202661 13256 raft_consensus.cc:2781] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:30.228412 11634 raft_consensus.cc:1226] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 3. Current term is 5. Ops: []
I20260809 15:52:30.228276 13356 raft_consensus.cc:729] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:30.228615 13256 raft_consensus.cc:686] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 3 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:30.229552 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.196192 13383 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:30.236758 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.197837 13234 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:30.198151 13359 raft_consensus.cc:686] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:30.238529 11637 raft_consensus.cc:2454] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:30.240111 11639 raft_consensus.cc:2454] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:52:30.249922 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:30.212334 13282 raft_consensus.cc:686] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:30.248502 13234 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
W20260809 15:52:30.259541 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 132935us
I20260809 15:52:30.256383 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.256735 10839 raft_consensus.cc:2454] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:30.258909 13321 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:30.259320 10893 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:30.249362 13289 raft_consensus.cc:2741] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260809 15:52:30.264719 11604 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:30.266150 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.266968 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.269598 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:30.262979 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:30.263851 13352 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:30.267937 13312 raft_consensus.cc:2726] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:30.272722 10811 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:30.283126 10842 raft_consensus.cc:1226] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:30.278501 10921 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:30.286276 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:30.286881 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.287159 10929 raft_consensus.cc:3037] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:30.287369 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.287528 10930 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:30.293298 11634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41478 (request call id 895) took 1038 ms. Trace:
I20260809 15:52:30.278332 13260 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:30.295332 11636 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:30.285199 13344 raft_consensus.cc:2781] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:52:30.317909 11011 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:30.271107 13333 consensus_queue.cc:1035] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.021s
I20260809 15:52:30.319597 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.320369 10842 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:30.273931 13354 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:30.332661 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 72375.2us
I20260809 15:52:30.333679 11604 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:30.334738 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 70627.4us
I20260809 15:52:30.313999 13060 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:30.322293 13312 raft_consensus.cc:2781] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:30.339404 13360 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:30.341821 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.321285 13344 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:30.353626 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.355549 11637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41482 (request call id 556) took 1534 ms. Trace:
I20260809 15:52:30.356026 11637 rpcz_store.cc:276] 0809 15:52:28.820729 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:28.821952 (+ 1223us) service_pool.cc:227] Handling call
0809 15:52:30.353351 (+1531399us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54004f3dd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:30.355479 (+ 2128us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2839498112}
I20260809 15:52:30.361166 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:30.346182 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:30.365935 10811 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:30.365136 10891 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:30.273770 13356 consensus_queue.cc:260] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.350174 13317 raft_consensus.cc:2781] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:30.354957 13344 raft_consensus.cc:3037] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
W20260809 15:52:30.357482 13393 log.cc:924] Time spent T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.120s user 0.001s sys 0.000s
I20260809 15:52:30.351030 13312 raft_consensus.cc:686] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:30.296790 13256 consensus_queue.cc:237] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.291852 13359 consensus_queue.cc:237] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.372690 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.368213 10842 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:30.319280 13282 consensus_queue.cc:237] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.361763 13207 raft_consensus.cc:482] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:30.377691 13336 raft_consensus.cc:2781] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:30.378386 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.371482 13317 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:30.345899 13243 raft_consensus.cc:2726] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:30.380128 10891 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:30.380841 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:30.379020 13260 raft_consensus.cc:2781] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:30.314153 11634 rpcz_store.cc:276] 0809 15:52:29.255046 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:29.443193 (+188147us) service_pool.cc:227] Handling call
0809 15:52:29.450717 (+ 7524us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:30.229055 (+778338us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:30.293029 (+ 63974us) spinlock_profiling.cc:243] Waited 757 ms on lock 0x7b540060e854. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:30.293236 (+ 207us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1741571328}
W20260809 15:52:30.385039 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 46727.1us
I20260809 15:52:30.372411 10920 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 4
I20260809 15:52:30.385252 13317 raft_consensus.cc:3037] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:30.403017 10811 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:30.403873 10893 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:30.405421 10893 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:30.390601 13312 consensus_queue.cc:237] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:30.406495 13391 log.cc:924] Time spent T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.129s user 0.001s sys 0.000s
I20260809 15:52:30.391255 13218 consensus_queue.cc:1046] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260809 15:52:30.393633 13395 log.cc:924] Time spent T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.078s user 0.000s sys 0.000s
I20260809 15:52:30.408604 13218 raft_consensus.cc:3032] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:30.408968 13218 raft_consensus.cc:729] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:30.409874 13218 consensus_queue.cc:260] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.410503 11015 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:30.411276 13218 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:30.412045 13260 raft_consensus.cc:686] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:30.413149 13260 consensus_queue.cc:237] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.380651 13356 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:30.365257 13360 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.376377 10930 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:30.398874 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.399401 13337 raft_consensus.cc:2781] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election won for term 4
W20260809 15:52:30.400436 13397 log.cc:924] Time spent T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.082s user 0.001s sys 0.000s
I20260809 15:52:30.386883 13207 raft_consensus.cc:504] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.430644 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:30.400650 10848 raft_consensus.cc:1261] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260809 15:52:30.432209 13207 leader_election.cc:288] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:30.432438 13167 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:52:30.432830 13167 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:30.433256 13167 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
W20260809 15:52:30.434368 13392 log.cc:924] Time spent T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.149s user 0.001s sys 0.000s
I20260809 15:52:30.435051 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:30.435312 13354 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:30.422921 13336 raft_consensus.cc:686] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:30.435849 11637 raft_consensus.cc:2379] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 4: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:30.436564 13336 consensus_queue.cc:237] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:30.437084 13318 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 15:52:30.424376 10930 raft_consensus.cc:595] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:30.438261 10930 raft_consensus.cc:603] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 15:52:30.438403 13337 raft_consensus.cc:686] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:30.387012 10838 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:30.415045 11018 raft_consensus.cc:2454] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:30.429369 13322 raft_consensus.cc:2781] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:52:30.429940 10893 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:30.441512 10929 raft_consensus.cc:2454] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:30.432701 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:30.433821 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:30.424849 10839 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:30.428879 10811 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:30.443403 11606 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:30.446136 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
I20260809 15:52:30.446691 11642 raft_consensus.cc:1799] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Handling vote request from an unknown peer d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:30.447197 11642 raft_consensus.cc:2379] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 5: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:30.450006 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:30.450729 11018 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 5
I20260809 15:52:30.393712 13361 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.442019 10921 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:30.455295 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.463263 10930 raft_consensus.cc:2379] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 5: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:30.456768 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.457165 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.462419 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:52:30.448858 13289 raft_consensus.cc:2781] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:30.470013 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.470242 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:30.470649 10839 raft_consensus.cc:2454] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:30.442728 13322 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:30.471364 13322 raft_consensus.cc:3037] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:30.474366 10893 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:30.476621 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.477186 11010 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:30.477880 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.479100 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.479620 13361 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:30.479885 10929 raft_consensus.cc:2379] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 3: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:30.480237 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.464121 10926 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:30.482004 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.482828 10923 raft_consensus.cc:2354] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5.
I20260809 15:52:30.483875 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.483749 13167 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.484748 10931 raft_consensus.cc:2379] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 3: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:30.470686 13326 raft_consensus.cc:2741] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260809 15:52:30.485761 13243 raft_consensus.cc:2781] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:30.485970 10922 raft_consensus.cc:1226] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 2. Current term is 4. Ops: []
I20260809 15:52:30.486382 13243 raft_consensus.cc:686] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:30.486893 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.487078 13352 raft_consensus.cc:2781] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:52:30.487520 13352 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:30.487411 13243 consensus_queue.cc:237] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.488096 13352 raft_consensus.cc:3037] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:30.488328 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.489558 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.490063 10844 raft_consensus.cc:1799] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Handling vote request from an unknown peer d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:30.490628 10844 raft_consensus.cc:2396] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260809 15:52:30.490458 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.487537 10839 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:30.491537 11641 raft_consensus.cc:2354] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5.
I20260809 15:52:30.470155 13289 raft_consensus.cc:686] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:30.499779 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.491472 13360 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:30.502069 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.502934 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.504732 10920 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:30.505646 13289 consensus_queue.cc:237] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.537097 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.507792 13167 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:30.500957 11636 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:30.541759 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.512266 10811 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:30.513321 10809 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:30.511569 13352 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:30.522202 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 136432us
I20260809 15:52:30.530210 11015 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:30.532298 10837 raft_consensus.cc:2354] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 2. Current term is 3.
I20260809 15:52:30.531457 11018 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:30.439366 13337 consensus_queue.cc:237] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:30.546980 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.547286 11604 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:30.547889 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.548517 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.549247 10924 raft_consensus.cc:2379] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 5: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:30.500475 13354 raft_consensus.cc:2781] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:30.551787 10840 raft_consensus.cc:2454] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:30.546841 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.550949 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.547541 10923 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:30.554224 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.558653 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:30.560650 10926 raft_consensus.cc:2454] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:30.562763 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:30.563409 11018 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:30.565068 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:30.566524 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:30.550877 13276 raft_consensus.cc:2781] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:30.568487 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:30.571709 11636 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:30.563171 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:30.571848 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 58605.2us
I20260809 15:52:30.573271 10811 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:30.555567 13354 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:30.574064 13276 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:30.576771 10811 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:30.596813 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:30.566520 13312 raft_consensus.cc:2726] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:30.575156 10930 raft_consensus.cc:2454] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:52:30.598903 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.576187 11604 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:30.600059 11606 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:30.600666 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:30.614599 10842 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:30.615653 10811 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:30.616981 10926 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:52:30.602415 11015 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:30.566305 13243 raft_consensus.cc:2726] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:30.604579 10922 raft_consensus.cc:1226] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:30.623335 13249 raft_consensus.cc:2781] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:30.605221 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.623797 13249 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:30.624191 13249 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:30.606554 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:30.625192 10929 raft_consensus.cc:2362] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5. Re-sending same reply.
I20260809 15:52:30.574149 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.626591 10921 raft_consensus.cc:2379] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 5: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:30.627182 13060 raft_consensus.cc:2781] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:30.627648 13060 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:30.628032 13060 raft_consensus.cc:3037] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:30.628700 13318 raft_consensus.cc:2781] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:30.629231 13318 raft_consensus.cc:686] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:30.630175 13318 consensus_queue.cc:237] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.630788 10923 raft_consensus.cc:1261] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 8. (index mismatch)
I20260809 15:52:30.632761 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:30.633553 11634 raft_consensus.cc:2354] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4.
I20260809 15:52:30.634786 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.637558 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:30.638336 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 65999.3us
I20260809 15:52:30.639165 10809 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:30.620473 10811 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:30.577371 13260 raft_consensus.cc:2781] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:30.598673 13269 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:30.601963 13354 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:30.643798 11015 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 8.
I20260809 15:52:30.602965 13276 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:30.649307 13269 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.651573 13269 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:30.561931 13383 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.619521 10978 leader_election.cc:398] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Vote denied by peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685) with higher term. Message: Invalid argument: T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5.
I20260809 15:52:30.644016 11018 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:30.647336 10809 leader_election.cc:302] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:30.656684 10978 leader_election.cc:401] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260809 15:52:30.622409 10891 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:30.653638 11604 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:30.654738 11635 raft_consensus.cc:2396] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260809 15:52:30.652297 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.658520 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:30.658969 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:30.659147 10840 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:30.659011 13233 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.620133 13369 raft_consensus.cc:2781] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:30.656415 10930 raft_consensus.cc:1226] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 3. Current term is 4. Ops: []
W20260809 15:52:30.667379 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 45058.7us
I20260809 15:52:30.665390 11606 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:30.655540 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:30.667851 11637 raft_consensus.cc:2379] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 3: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:30.669901 10929 raft_consensus.cc:1261] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260809 15:52:30.644493 13260 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:30.657955 13312 raft_consensus.cc:2781] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:30.673218 13312 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:30.674256 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:30.676098 10891 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:30.677811 11604 leader_election.cc:302] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:30.681267 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 82134.7us
I20260809 15:52:30.669132 13321 raft_consensus.cc:2741] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260809 15:52:30.640928 13060 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.674068 13260 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:30.687647 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:30.688561 11604 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:30.687954 10891 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:30.648236 13317 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.676751 13361 raft_consensus.cc:2726] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 15:52:30.690191 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 69765.5us
I20260809 15:52:30.689378 11632 raft_consensus.cc:1226] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260809 15:52:30.679755 13356 raft_consensus.cc:2781] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:30.673873 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:30.681668 13269 raft_consensus.cc:2726] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:30.691883 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:30.692977 11639 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:30.692371 10891 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:30.689704 13360 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 5
I20260809 15:52:30.685477 13312 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:30.686684 13417 raft_consensus.cc:2781] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:30.688340 13289 raft_consensus.cc:2781] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:30.698278 13232 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:30.699940 13317 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:30.696751 13356 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:30.717924 13356 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:30.707365 13326 raft_consensus.cc:2781] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:30.718194 13232 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:30.718670 13326 raft_consensus.cc:686] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:30.718801 13232 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:30.719630 13326 consensus_queue.cc:237] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:30.685953 13233 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:30.703195 10811 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:30.710592 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:30.693043 13321 raft_consensus.cc:2781] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:30.723809 11015 raft_consensus.cc:1261] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:30.724560 10842 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:30.726416 13383 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:30.694239 13352 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:30.737211 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:30.714689 13289 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 15:52:30.737789 13334 consensus_meta.cc:287] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.532s user 0.005s sys 0.005s
I20260809 15:52:30.749204 13289 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:30.751009 11636 raft_consensus.cc:1226] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:30.756042 11639 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
W20260809 15:52:30.757175 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 109942us
I20260809 15:52:30.767819 13289 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:30.758001 10809 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:30.757050 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:30.746039 13321 raft_consensus.cc:686] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:30.747213 13416 raft_consensus.cc:2781] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:30.776017 13369 raft_consensus.cc:686] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:30.781731 13369 consensus_queue.cc:237] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.796828 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
W20260809 15:52:30.798243 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 68771.4us
W20260809 15:52:30.799180 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 58864.8us
I20260809 15:52:30.799453 10809 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:30.797044 10978 leader_election.cc:398] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Vote denied by peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783) with higher term. Message: Invalid argument: T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5.
I20260809 15:52:30.805886 10978 leader_election.cc:401] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
W20260809 15:52:30.800601 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 115185us
I20260809 15:52:30.805191 10811 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:30.749854 13276 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:30.754451 13167 consensus_queue.cc:1035] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260809 15:52:30.754561 13334 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.777992 13416 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:30.802706 13417 raft_consensus.cc:686] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:30.809149 13060 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:30.818235 10809 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:30.801901 13322 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.816359 13218 raft_consensus.cc:2726] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783) with higher term. Message: Invalid argument: T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5.
I20260809 15:52:30.816262 13416 raft_consensus.cc:3037] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:30.816042 13344 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.850396 13289 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:30.822486 13321 consensus_queue.cc:237] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:30.834893 13419 raft_consensus.cc:2781] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:30.830853 11604 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:30.842588 13417 consensus_queue.cc:237] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:30.875737 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:30.876477 10838 raft_consensus.cc:3037] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 3
I20260809 15:52:30.876256 13356 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.878346 13356 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:30.892959 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
W20260809 15:52:30.869451 13234 consensus_meta.cc:287] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.596s user 0.010s sys 0.000s
I20260809 15:52:30.900358 11606 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:30.868762 13428 raft_consensus.cc:2726] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:30.904353 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:30.884651 13419 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:30.910115 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:30.910319 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:30.910702 11014 raft_consensus.cc:3037] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:30.910832 11009 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:30.913028 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:30.913586 11010 raft_consensus.cc:3037] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:30.915021 11011 raft_consensus.cc:1261] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:52:30.916330 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:30.890512 13382 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:30.905002 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:30.919622 10930 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:30.921763 10921 raft_consensus.cc:1226] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:30.922842 13425 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:30.923449 13425 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.924674 13269 raft_consensus.cc:2781] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:30.925282 13269 raft_consensus.cc:686] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:30.925734 13425 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:30.903695 13234 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.926647 13421 raft_consensus.cc:2781] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:30.926321 13269 consensus_queue.cc:237] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:30.927099 13414 log.cc:924] Time spent T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.133s user 0.001s sys 0.000s
I20260809 15:52:30.927399 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:30.927209 13382 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:30.902519 13218 consensus_queue.cc:1046] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260809 15:52:30.927860 13299 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.048s
I20260809 15:52:30.932236 13232 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:30.917557 13419 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:30.934608 13422 raft_consensus.cc:2726] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:30.979732 13234 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:30.980602 11636 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:30.935464 13260 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:30.983143 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:30.898798 13289 raft_consensus.cc:2781] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:52:30.983928 13260 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:30.984236 13289 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:30.939217 13344 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:30.940033 13276 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:30.942660 13312 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:30.897993 13233 raft_consensus.cc:2726] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:30.945884 11010 raft_consensus.cc:2454] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:30.963152 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:30.996722 11011 raft_consensus.cc:3037] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:30.971743 11606 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:30.952863 13421 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:31.020107 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:30.983399 10841 raft_consensus.cc:2379] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 5: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:31.020947 10840 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
W20260809 15:52:30.936820 13420 log.cc:924] Time spent T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.064s user 0.000s sys 0.000s
I20260809 15:52:30.942329 11014 raft_consensus.cc:2454] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:30.987237 11009 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
W20260809 15:52:31.020427 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 48767.2us
W20260809 15:52:31.036587 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 218410us
I20260809 15:52:30.988428 13207 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:31.046272 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:31.046954 10843 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:31.053824 13218 raft_consensus.cc:3032] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:31.035482 13421 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:31.054322 13218 raft_consensus.cc:729] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:31.055131 13218 consensus_queue.cc:260] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:31.055756 10843 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 7.
I20260809 15:52:31.056404 13218 raft_consensus.cc:3037] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:31.059645 13382 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:31.061759 10838 raft_consensus.cc:2454] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:31.040038 11636 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 8.
I20260809 15:52:31.041122 10930 raft_consensus.cc:2454] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:31.043332 10809 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:30.984689 13289 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:30.985991 13322 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:30.988668 13334 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:31.080111 11606 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:31.035843 13318 raft_consensus.cc:2726] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:31.103066 11606 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
W20260809 15:52:31.104521 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 61276.5us
I20260809 15:52:31.115062 10809 leader_election.cc:302] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:31.150864 13382 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:31.151427 13382 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:31.153481 13382 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:31.126060 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 321175us
I20260809 15:52:31.104185 13060 raft_consensus.cc:2781] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:31.106891 13322 raft_consensus.cc:2741] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260809 15:52:31.121074 13356 raft_consensus.cc:2781] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:31.156047 10840 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:31.126163 13232 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:31.132443 13430 log.cc:924] Time spent T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.079s user 0.001s sys 0.000s
I20260809 15:52:31.147759 11011 raft_consensus.cc:2454] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:31.161626 11010 raft_consensus.cc:1226] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:31.148236 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:31.162964 10930 raft_consensus.cc:2354] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for earlier term 7. Current term is 8.
W20260809 15:52:31.164227 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 49285.6us
I20260809 15:52:31.104993 13207 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:31.169173 10809 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:31.177266 13416 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:31.171568 13060 raft_consensus.cc:686] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 3 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
W20260809 15:52:31.183415 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 378296us
W20260809 15:52:31.168315 13249 consensus_meta.cc:287] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.544s user 0.009s sys 0.002s
I20260809 15:52:31.187456 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:31.192893 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:31.174263 13260 raft_consensus.cc:2726] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:31.202507 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:31.178063 13207 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:31.178532 13312 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:31.206657 11016 raft_consensus.cc:1261] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260809 15:52:31.211112 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:31.211683 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:31.213606 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 58873us
I20260809 15:52:31.197722 13428 raft_consensus.cc:2726] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:31.219878 11010 raft_consensus.cc:2379] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 3: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:31.212258 11013 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:31.200759 13344 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:31.168617 13354 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:31.230142 10978 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:31.228791 10811 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:31.228756 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:31.203956 13167 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:31.199863 13249 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:31.325419 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:31.328676 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:31.329972 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:31.328153 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
W20260809 15:52:31.328138 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 104367us
I20260809 15:52:31.324404 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:31.350639 10840 raft_consensus.cc:2379] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 5: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:31.338439 11604 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
W20260809 15:52:31.343582 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 105867us
I20260809 15:52:31.348105 10847 raft_consensus.cc:1226] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:31.353899 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:31.355161 11635 raft_consensus.cc:1226] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:31.356069 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:31.233135 13425 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:31.429399 10844 raft_consensus.cc:2354] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6.
I20260809 15:52:31.349728 11641 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 3 LEARNER]: Advancing to term 5
I20260809 15:52:31.430275 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:31.422760 10840 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:45594
I20260809 15:52:31.432765 10840 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:31.433203 10840 raft_consensus.cc:3037] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:31.232524 13276 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:31.418414 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:31.325942 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:31.421173 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:52:31.421806 13432 log.cc:924] Time spent T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.221s user 0.000s sys 0.001s
I20260809 15:52:31.328241 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:31.228471 13356 raft_consensus.cc:686] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:31.422195 10837 raft_consensus.cc:1226] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:31.422484 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:31.461953 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:31.418478 10931 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
W20260809 15:52:31.418781 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 190075us
I20260809 15:52:31.462581 10839 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:31.424904 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:31.425549 10843 raft_consensus.cc:2379] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 4: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:31.425705 11009 raft_consensus.cc:2454] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:31.426000 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:31.464635 11010 raft_consensus.cc:2379] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 5: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:31.222280 13336 consensus_queue.cc:1046] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.134s
I20260809 15:52:31.358333 13428 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:31.425426 11606 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:31.228995 13282 consensus_queue.cc:1035] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.194s
I20260809 15:52:31.335888 13344 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:31.420571 13167 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:31.454154 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:31.454057 13425 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:31.328418 13445 raft_consensus.cc:2781] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:31.477226 13445 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:31.473078 13428 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:31.455212 10978 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:31.477607 10838 raft_consensus.cc:2379] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 3: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:31.455034 13326 consensus_queue.cc:1035] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:31.456254 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:31.458772 11634 raft_consensus.cc:1261] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:31.430389 13060 consensus_queue.cc:237] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:31.460816 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:31.480592 11018 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 4
I20260809 15:52:31.480916 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:31.481029 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:31.462013 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:31.463053 10811 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:31.481639 10927 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:31.426712 13260 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:31.422199 13269 raft_consensus.cc:2781] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:31.423228 13233 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:31.452008 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:31.491050 10930 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:31.470726 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:31.471940 10840 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:31.475600 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 240610us
W20260809 15:52:31.475418 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 50102.3us
I20260809 15:52:31.479130 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:31.491902 10847 raft_consensus.cc:2379] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 5: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:31.492725 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:31.492884 10840 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:31.493718 13436 log.cc:924] Time spent T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.135s user 0.000s sys 0.000s
I20260809 15:52:31.495301 13356 consensus_queue.cc:237] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:31.498263 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:31.467578 11011 raft_consensus.cc:2932] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Committing config change with OpId 1.3: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:31.499121 13167 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:31.479144 13232 raft_consensus.cc:2781] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:31.482383 13425 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:31.502187 13269 raft_consensus.cc:686] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 3 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:31.502804 13232 raft_consensus.cc:686] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:31.503248 13269 consensus_queue.cc:237] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:31.503783 13232 consensus_queue.cc:237] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:31.506507 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:31.507153 10842 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:31.479372 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:31.481813 10923 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:31.485515 13312 raft_consensus.cc:3037] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:31.461396 13276 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:31.511210 13344 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:31.470120 11640 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
W20260809 15:52:31.474256 13360 consensus_meta.cc:287] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.760s user 0.008s sys 0.003s
I20260809 15:52:31.485425 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:31.485443 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:31.486464 11010 raft_consensus.cc:1261] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 15:52:31.523885 11009 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 6
I20260809 15:52:31.486825 10839 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 8.
I20260809 15:52:31.503433 13289 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:31.475952 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:31.493891 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:31.537274 10837 raft_consensus.cc:2454] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:31.537671 10840 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:31.492951 10926 raft_consensus.cc:1261] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:31.513224 10847 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:31.513670 10978 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:31.515491 11606 leader_election.cc:398] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Vote denied by peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783) with higher term. Message: Invalid argument: T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for earlier term 7. Current term is 8.
I20260809 15:52:31.538504 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:31.513532 10927 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:31.539088 11016 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:31.532343 11641 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 5 LEARNER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:31.523396 10848 raft_consensus.cc:2454] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:31.533111 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:31.483883 13444 raft_consensus.cc:2781] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:31.540323 11015 raft_consensus.cc:2379] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 5: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:31.477638 13445 raft_consensus.cc:3037] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:31.538703 11606 leader_election.cc:401] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260809 15:52:31.520334 10842 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:31.542900 11009 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:31.522876 13360 raft_consensus.cc:2726] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685) with higher term. Message: Invalid argument: T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5.
I20260809 15:52:31.557170 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
W20260809 15:52:31.565157 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 51555.4us
I20260809 15:52:31.565479 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:31.566156 10978 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:31.566423 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:31.567139 11010 raft_consensus.cc:2379] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 5: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:31.566164 10843 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:31.571760 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:31.572857 11009 raft_consensus.cc:2454] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:31.561415 10930 raft_consensus.cc:2454] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:31.549889 11018 raft_consensus.cc:2454] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:31.490617 13416 leader_election.cc:288] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:31.557199 13421 consensus_meta.cc:287] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.502s user 0.008s sys 0.003s
I20260809 15:52:31.576524 10811 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:31.577358 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:31.582528 11016 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:31.583725 13426 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:31.585474 13426 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:31.587728 13426 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:31.580551 13282 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:31.588907 13282 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:31.590181 13344 consensus_queue.cc:1035] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:52:31.590904 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:31.591019 13282 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:31.591758 10930 raft_consensus.cc:2454] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:31.593694 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
W20260809 15:52:31.593050 13453 log.cc:924] Time spent T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.081s user 0.000s sys 0.001s
I20260809 15:52:31.603307 13289 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:31.604823 13416 raft_consensus.cc:2726] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 15:52:31.578491 13317 raft_consensus.cc:2726] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783) with higher term. Message: Invalid argument: T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for earlier term 7. Current term is 8.
I20260809 15:52:31.606721 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:31.596969 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:31.607432 10927 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:31.617604 11017 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:31.620271 13421 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:31.762590 10841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 1915) took 1502 ms. Trace:
I20260809 15:52:31.743096 13455 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:31.779284 10841 rpcz_store.cc:276] 0809 15:52:30.260254 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:30.260435 (+ 181us) service_pool.cc:227] Handling call
0809 15:52:31.762311 (+1501876us) spinlock_profiling.cc:243] Waited 716 ms on lock 0x7b54001c61d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:31.762478 (+ 167us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1647733632}
W20260809 15:52:31.744091 13419 consensus_meta.cc:287] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.770s user 0.011s sys 0.000s
I20260809 15:52:31.725420 11640 raft_consensus.cc:2454] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:31.736544 13243 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:31.722934 13460 raft_consensus.cc:482] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:31.673797 13422 consensus_queue.cc:1035] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260809 15:52:31.687047 13276 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:31.722229 13442 log.cc:924] Time spent T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.050s user 0.000s sys 0.001s
I20260809 15:52:31.706584 10893 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:31.623634 11010 raft_consensus.cc:3037] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
W20260809 15:52:31.708457 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 142430us
I20260809 15:52:31.658427 10809 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:31.798950 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54224 (request call id 1841) took 1264 ms. Trace:
W20260809 15:52:31.800668 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 142331us
I20260809 15:52:31.800731 10978 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:31.779515 13455 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:31.723791 13218 consensus_meta.cc:287] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.667s user 0.012s sys 0.001s
I20260809 15:52:31.800329 10931 rpcz_store.cc:276] 0809 15:52:30.534149 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:30.534313 (+ 164us) service_pool.cc:227] Handling call
0809 15:52:31.798834 (+1264521us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b54000a91d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:31.798905 (+ 71us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1990635776}
I20260809 15:52:31.528357 13233 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:31.694231 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:31.807570 10926 raft_consensus.cc:3032] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:31.807878 10926 raft_consensus.cc:729] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:31.658578 10848 raft_consensus.cc:1261] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:52:31.698938 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
W20260809 15:52:31.600401 13450 log.cc:924] Time spent T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.089s user 0.000s sys 0.002s
I20260809 15:52:31.700518 13249 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:31.639658 11606 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
W20260809 15:52:31.812666 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 173097us
I20260809 15:52:31.812774 13256 raft_consensus.cc:2781] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:31.813225 13256 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:31.813537 13256 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:31.814965 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: true dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:31.669742 10839 raft_consensus.cc:1261] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260809 15:52:31.815920 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:31.816062 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:31.816941 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:31.608978 13360 raft_consensus.cc:2741] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260809 15:52:31.581100 13428 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:31.817588 11014 raft_consensus.cc:2354] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for earlier term 7. Current term is 8.
I20260809 15:52:31.544060 13444 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:31.817992 13444 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
W20260809 15:52:31.818416 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 111926us
I20260809 15:52:31.819443 10893 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:31.820886 11604 leader_election.cc:398] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Vote denied by peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223) with higher term. Message: Invalid argument: T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for earlier term 7. Current term is 8.
W20260809 15:52:31.641904 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 59416us
I20260809 15:52:31.738757 11016 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:31.823546 10841 raft_consensus.cc:1261] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:52:31.824680 10811 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:31.764789 13321 raft_consensus.cc:3032] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:31.761694 13312 raft_consensus.cc:2726] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:31.693608 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:31.709856 13354 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:31.618662 10923 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:31.619736 10847 raft_consensus.cc:2454] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:31.578064 13382 raft_consensus.cc:3037] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:31.723838 13260 raft_consensus.cc:2726] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:31.614313 10891 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:31.782029 10844 raft_consensus.cc:2379] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 4: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:31.794701 10734 catalog_manager.cc:5495] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 3 to 4, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 4 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:31.787909 13243 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:31.829154 10926 consensus_queue.cc:260] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:31.674640 10840 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:31.860350 11014 raft_consensus.cc:1261] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:31.837406 11016 raft_consensus.cc:1261] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260809 15:52:31.830116 11013 raft_consensus.cc:2454] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:31.803869 10737 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33038 (request call id 392) took 3124 ms. Trace:
I20260809 15:52:31.808457 11012 raft_consensus.cc:3037] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:31.863651 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:31.864508 11640 raft_consensus.cc:2379] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 4: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:31.799175 13460 raft_consensus.cc:504] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:31.809643 13465 raft_consensus.cc:2741] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260809 15:52:31.789000 10922 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:31.869725 13243 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:31.874212 13382 raft_consensus.cc:2726] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:31.875028 11015 raft_consensus.cc:1261] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:31.875241 13427 raft_consensus.cc:2726] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:31.876605 11015 raft_consensus.cc:1261] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:31.877782 11015 raft_consensus.cc:1261] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260809 15:52:31.878527 13269 consensus_queue.cc:1035] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:31.810709 13233 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:31.811319 11637 raft_consensus.cc:1261] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:31.647193 13317 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:31.830418 10978 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:31.898330 13317 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:31.898931 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 68590.4us
I20260809 15:52:31.899819 13243 consensus_queue.cc:1035] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:31.899933 10978 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:31.900918 13382 raft_consensus.cc:2726] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:31.901036 10978 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:31.901515 13382 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:31.832232 10811 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:31.833562 11010 raft_consensus.cc:1261] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:31.805951 13322 raft_consensus.cc:2726] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:31.903318 13322 raft_consensus.cc:2781] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:31.903790 13322 raft_consensus.cc:686] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:31.904790 13465 raft_consensus.cc:2741] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260809 15:52:31.904621 13322 consensus_queue.cc:237] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:31.834753 11636 raft_consensus.cc:1261] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:31.797541 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:31.906553 11017 raft_consensus.cc:2379] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 3: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:31.860870 10926 raft_consensus.cc:3037] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:31.904644 11010 raft_consensus.cc:1261] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:52:31.909617 10978 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:31.803308 10809 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:31.910558 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 107291us
I20260809 15:52:31.910761 10978 leader_election.cc:398] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Vote denied by peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) with higher term. Message: Invalid argument: T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 2. Current term is 3.
I20260809 15:52:31.911041 10978 leader_election.cc:401] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260809 15:52:31.911216 10809 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:31.911566 13218 raft_consensus.cc:2726] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:31.912351 13276 raft_consensus.cc:2726] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:31.912622 10809 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:31.912925 13276 raft_consensus.cc:2726] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:31.914029 10978 leader_election.cc:302] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:31.914855 10809 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:31.879009 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:31.866281 13460 leader_election.cc:288] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:31.825794 13431 raft_consensus.cc:2781] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:31.836265 10930 raft_consensus.cc:1261] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260809 15:52:31.881157 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 266942us
I20260809 15:52:31.882579 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:31.830667 13321 raft_consensus.cc:729] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:31.885267 11641 raft_consensus.cc:1261] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260809 15:52:31.891708 13467 log.cc:924] Time spent T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.111s user 0.001s sys 0.000s
W20260809 15:52:31.893271 13446 log.cc:924] Time spent T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.398s user 0.000s sys 0.003s
I20260809 15:52:31.895327 10837 raft_consensus.cc:1261] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 15:52:31.913949 13425 raft_consensus.cc:2726] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:31.898926 13299 consensus_queue.cc:1035] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.003s
W20260809 15:52:31.902462 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 70295.5us
I20260809 15:52:31.915061 10978 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:31.787066 13419 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:31.810151 13344 raft_consensus.cc:2726] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:31.917797 10809 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:31.916844 11014 raft_consensus.cc:2454] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:31.879379 11015 raft_consensus.cc:1261] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260809 15:52:31.978845 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 61135.6us
I20260809 15:52:31.979538 10809 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:31.921679 10893 leader_election.cc:398] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Vote denied by peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) with higher term. Message: Invalid argument: T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4.
I20260809 15:52:31.920089 11012 raft_consensus.cc:2454] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:31.882695 10843 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:31.981299 10893 leader_election.cc:401] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260809 15:52:31.881106 13354 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:31.825788 13249 consensus_queue.cc:1035] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.098s
I20260809 15:52:31.922963 13276 raft_consensus.cc:2726] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:31.862581 10737 rpcz_store.cc:276] 0809 15:52:28.674222 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:28.674374 (+ 152us) service_pool.cc:227] Handling call
0809 15:52:29.184013 (+509639us) spinlock_profiling.cc:243] Waited 127 ms on lock 0x7b5400303754. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478af0 00007f79694fbf2a 00007f796f44dd6c 00007f796f547c3e 00007f796d859fb4 00007f796d859f42 00007f796d859e71 00007f796d859dec 00007f796d8590a2 00007f796791d94c 00007f796791cd86
0809 15:52:31.799029 (+2615016us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 15:52:29.955823 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 15:52:30.174226 (+218403us) write_op.cc:432] Acquiring schema lock in shared mode
0809 15:52:30.185117 (+ 10891us) write_op.cc:435] Acquired schema lock
0809 15:52:30.195745 (+ 10628us) tablet.cc:660] Decoding operations
0809 15:52:30.264992 (+ 69247us) write_op.cc:620] Acquiring the partition lock for write op
0809 15:52:30.275347 (+ 10355us) write_op.cc:641] Partition lock acquired for write op
0809 15:52:30.276950 (+ 1603us) tablet.cc:683] Acquiring locks for 1 operations
0809 15:52:30.362882 (+ 85932us) tablet.cc:699] Row locks acquired
0809 15:52:30.368671 (+ 5789us) write_op.cc:260] PREPARE: finished
0809 15:52:30.379727 (+ 11056us) write_op.cc:270] Start()
0809 15:52:30.407250 (+ 27523us) write_op.cc:276] Timestamp: P: 1786290750375233 usec, L: 0
0809 15:52:30.407307 (+ 57us) op_driver.cc:331] REPLICATION: starting
0809 15:52:30.408025 (+ 718us) log.cc:842] Serialized 527 byte log entry
0809 15:52:30.522491 (+114466us) op_driver.cc:444] REPLICATION: finished
0809 15:52:30.687696 (+165205us) write_op.cc:301] APPLY: starting
0809 15:52:30.692155 (+ 4459us) tablet.cc:1365] starting BulkCheckPresence
0809 15:52:30.724964 (+ 32809us) tablet.cc:1368] finished BulkCheckPresence
0809 15:52:30.747087 (+ 22123us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 15:52:30.809335 (+ 62248us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 15:52:30.838420 (+ 29085us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 15:52:30.838499 (+ 79us) write_op.cc:312] APPLY: finished
0809 15:52:30.878896 (+ 40397us) log.cc:842] Serialized 23 byte log entry
0809 15:52:30.903355 (+ 24459us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:52:30.908440 (+ 5085us) write_op.cc:454] Released schema lock
0809 15:52:30.937196 (+ 28756us) write_op.cc:341] FINISH: Updating metrics
0809 15:52:30.937615 (+ 419us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:52:30.937641 (+ 26us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":198246,"reported_tablets":1,"spinlock_wait_cycles":292024576,"child_traces":[["op",{"apply.queue_time_us":144372,"apply.run_cpu_time_us":3419,"apply.run_wall_time_us":259367,"num_ops":1,"prepare.queue_time_us":81272,"prepare.run_cpu_time_us":14024,"prepare.run_wall_time_us":537287,"raft.queue_time_us":1086,"raft.run_cpu_time_us":20665,"raft.run_wall_time_us":81564,"replication_time_us":94390,"spinlock_wait_cycles":1408,"thread_start_us":202451,"threads_started":4,"wal-append.queue_time_us":57557}]]}
I20260809 15:52:31.886924 13167 raft_consensus.cc:2781] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:31.937863 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:31.934679 10926 raft_consensus.cc:2454] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
W20260809 15:52:31.973281 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 58282.7us
I20260809 15:52:31.973574 11016 raft_consensus.cc:1261] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 15:52:31.975003 10811 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:31.983916 10809 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:31.985649 10931 raft_consensus.cc:1261] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:31.968797 13382 raft_consensus.cc:2726] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:31.961319 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:31.987161 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:31.987776 11604 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:32.005401 10811 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:31.994390 10978 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:31.995883 11014 raft_consensus.cc:1261] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 15:52:32.005920 10840 raft_consensus.cc:1261] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260809 15:52:31.978303 13465 raft_consensus.cc:2726] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:31.982597 13243 consensus_queue.cc:1035] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.017s
I20260809 15:52:31.999059 13256 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:31.987007 13425 raft_consensus.cc:2726] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
W20260809 15:52:32.001996 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 117566us
I20260809 15:52:31.942888 13431 raft_consensus.cc:686] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:31.990736 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:32.010262 10926 raft_consensus.cc:1261] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:32.023983 10978 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:31.989424 13312 raft_consensus.cc:2781] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:31.982894 13218 raft_consensus.cc:2726] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) with higher term. Message: Invalid argument: T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 2. Current term is 3.
I20260809 15:52:31.935820 13383 consensus_queue.cc:1035] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.057s
I20260809 15:52:32.030895 10978 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:31.993359 13167 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:32.004058 13416 raft_consensus.cc:2781] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:32.037460 10978 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:32.007302 13276 raft_consensus.cc:2781] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:32.008316 13352 raft_consensus.cc:2726] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) with higher term. Message: Invalid argument: T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4.
I20260809 15:52:31.991513 13428 consensus_queue.cc:1035] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260809 15:52:32.029871 10922 raft_consensus.cc:2379] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 4: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:32.012130 13421 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:32.023370 13382 raft_consensus.cc:2726] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:32.033705 13465 raft_consensus.cc:2781] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:32.040123 13447 raft_consensus.cc:2781] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:32.042793 13167 raft_consensus.cc:3037] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:31.990780 13354 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:32.043108 13360 raft_consensus.cc:2781] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:32.042620 13422 raft_consensus.cc:2781] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:32.045526 13207 raft_consensus.cc:2781] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:32.056866 13470 raft_consensus.cc:2741] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260809 15:52:32.052942 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 47608us
I20260809 15:52:32.065062 13317 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:32.065805 13431 consensus_queue.cc:237] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:32.068759 13455 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:32.069610 13419 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:32.012853 13444 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:32.008013 13321 consensus_queue.cc:260] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:32.054025 13233 consensus_queue.cc:1035] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.017s
I20260809 15:52:32.049412 13448 consensus_queue.cc:1035] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260809 15:52:32.043229 13361 consensus_queue.cc:1035] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.007s
I20260809 15:52:32.067481 11606 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:32.022818 13289 consensus_queue.cc:1035] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.026s
I20260809 15:52:32.039528 13416 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:32.042240 13312 raft_consensus.cc:686] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:32.044124 13276 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:32.054880 13465 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:32.030200 13260 consensus_queue.cc:1035] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260809 15:52:32.134148 13321 raft_consensus.cc:3037] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:32.135689 13444 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:32.111930 13422 raft_consensus.cc:686] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:32.135810 13312 consensus_queue.cc:237] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:32.136803 13422 consensus_queue.cc:237] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:32.122366 13282 raft_consensus.cc:2726] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:32.096189 10811 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:32.074359 13447 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:32.127561 13276 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:32.138614 13447 raft_consensus.cc:3037] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:32.139134 10811 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:32.097438 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 86580.2us
I20260809 15:52:32.140810 10809 leader_election.cc:302] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:32.045423 13460 consensus_queue.cc:1046] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260809 15:52:32.112761 11012 raft_consensus.cc:1226] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 2. Current term is 3. Ops: []
W20260809 15:52:32.120422 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 86961.4us
W20260809 15:52:32.121301 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 56833.1us
I20260809 15:52:32.087615 13360 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:32.091562 13207 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:32.126868 11011 raft_consensus.cc:1226] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4. Ops: []
W20260809 15:52:32.127327 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 87517us
I20260809 15:52:32.129518 13256 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:32.124599 13416 raft_consensus.cc:3037] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:32.148521 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:32.153650 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 120804us
I20260809 15:52:32.157373 11014 raft_consensus.cc:1261] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:32.160378 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:32.160835 10840 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:32.164369 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:32.165025 11014 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:32.166814 10978 leader_election.cc:302] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:32.170442 10840 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:32.172458 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:32.173161 11013 raft_consensus.cc:2379] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 4: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:32.171274 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:32.171057 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:32.175052 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:32.139930 13465 raft_consensus.cc:3037] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:32.162431 13428 raft_consensus.cc:2781] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:32.154392 13421 raft_consensus.cc:2726] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 15:52:32.179771 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:32.180406 10922 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:32.126497 13269 consensus_queue.cc:1035] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.040s
I20260809 15:52:32.181365 13167 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:32.150261 13207 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:32.145120 13360 raft_consensus.cc:3037] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:32.145423 13260 raft_consensus.cc:2781] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:32.187394 10893 leader_election.cc:398] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Vote denied by peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) with higher term. Message: Invalid argument: T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6.
I20260809 15:52:32.192276 10893 leader_election.cc:401] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Cancelling election due to peer responding with higher term
I20260809 15:52:32.127072 13356 consensus_queue.cc:1035] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260809 15:52:32.154831 13444 raft_consensus.cc:3032] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:32.190620 13167 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:32.221873 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:32.150396 13299 consensus_queue.cc:1035] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260809 15:52:32.191676 13322 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:32.229681 10893 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:32.195451 13445 consensus_meta.cc:287] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.626s user 0.008s sys 0.002s
I20260809 15:52:32.222579 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:32.225214 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:32.188195 13218 raft_consensus.cc:2726] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:32.249156 10926 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:32.222584 11641 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:32.192024 13428 raft_consensus.cc:686] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:32.247179 10811 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:32.197645 13444 raft_consensus.cc:729] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:32.183516 13382 consensus_queue.cc:1035] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.055s
I20260809 15:52:32.200255 13260 raft_consensus.cc:686] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:32.165993 13417 consensus_queue.cc:1035] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.057s
I20260809 15:52:32.199671 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:32.223111 13449 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:32.230320 13424 raft_consensus.cc:2726] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: Vote denied by peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) with higher term. Message: Invalid argument: T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6.
I20260809 15:52:32.186962 13427 consensus_queue.cc:1035] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260809 15:52:32.227788 13233 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:32.272624 13422 raft_consensus.cc:2726] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:32.267928 13167 raft_consensus.cc:2726] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:32.274201 13481 consensus_queue.cc:1046] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.016s
I20260809 15:52:32.275986 11604 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:32.225209 13426 consensus_queue.cc:1046] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.036s
I20260809 15:52:32.287738 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:32.281430 10926 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:32.197855 13321 consensus_queue.cc:1046] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.058s
I20260809 15:52:32.315285 11641 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:32.325451 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:32.229904 13419 consensus_queue.cc:1035] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
W20260809 15:52:32.337666 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 61756.1us
I20260809 15:52:32.254032 13445 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:32.275267 13232 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:32.259876 13478 consensus_queue.cc:1035] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.066s
I20260809 15:52:32.291592 13354 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:32.251706 13322 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:32.349454 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 102342us
I20260809 15:52:32.373114 10811 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:32.406469 11018 raft_consensus.cc:1226] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
W20260809 15:52:32.418895 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 218045us
I20260809 15:52:32.275791 13449 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:32.385931 13425 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:32.385700 13299 raft_consensus.cc:2781] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:32.327190 13233 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:32.339948 13282 consensus_queue.cc:1035] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.088s
I20260809 15:52:32.538996 10843 raft_consensus.cc:1226] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:32.318979 13482 consensus_queue.cc:1035] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.190s
I20260809 15:52:32.512938 13426 raft_consensus.cc:3032] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:32.517216 13477 consensus_queue.cc:1035] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.250s
W20260809 15:52:32.518967 13476 log.cc:924] Time spent T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.265s user 0.000s sys 0.001s
W20260809 15:52:32.519969 13485 log.cc:924] Time spent T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.174s user 0.001s sys 0.000s
I20260809 15:52:32.520362 13435 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:32.530537 13483 log.cc:924] Time spent T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.255s user 0.000s sys 0.001s
W20260809 15:52:32.534407 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 310581us
W20260809 15:52:32.545400 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 172384us
I20260809 15:52:32.510440 13299 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:32.346678 13428 consensus_queue.cc:237] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:32.543720 13426 raft_consensus.cc:729] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:32.569092 13421 raft_consensus.cc:2726] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 15:52:32.374421 13232 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:32.606191 10893 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:32.611088 10893 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:32.592782 10811 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:32.385776 13260 consensus_queue.cc:237] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:32.536849 13425 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:32.601487 13299 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:32.579398 10978 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
W20260809 15:52:32.626128 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 356582us
W20260809 15:52:32.631830 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 65415.5us
I20260809 15:52:32.585444 13416 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:32.616264 13417 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 15:52:32.630987 13352 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:32.634260 13312 raft_consensus.cc:2781] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:32.657024 11011 raft_consensus.cc:1261] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 15:52:32.707929 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:52:32.715008 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 103990us
I20260809 15:52:32.651185 13477 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:32.595911 13435 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:32.708344 13447 consensus_meta.cc:287] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.569s user 0.009s sys 0.001s
W20260809 15:52:32.713665 13276 consensus_meta.cc:287] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.575s user 0.005s sys 0.002s
I20260809 15:52:32.825680 11017 raft_consensus.cc:1226] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:32.833784 10893 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:32.636142 13423 consensus_queue.cc:1046] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.468s
I20260809 15:52:32.708596 13489 raft_consensus.cc:2726] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260809 15:52:32.799981 13468 log.cc:924] Time spent T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.061s user 0.000s sys 0.000s
I20260809 15:52:32.852723 10893 leader_election.cc:398] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Vote denied by peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685) with higher term. Message: Invalid argument: T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4.
W20260809 15:52:32.825475 13360 consensus_meta.cc:287] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.627s user 0.010s sys 0.010s
I20260809 15:52:32.804662 13445 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:32.825508 13322 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:32.830343 13417 raft_consensus.cc:686] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 8 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:32.863561 10893 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:32.871594 13477 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:32.870573 13354 raft_consensus.cc:2781] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election won for term 4
W20260809 15:52:32.902315 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 51914.8us
I20260809 15:52:32.897617 13511 consensus_queue.cc:1046] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.043s
I20260809 15:52:32.870558 13494 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:32.874445 11017 raft_consensus.cc:1261] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 15:52:32.853615 13312 raft_consensus.cc:686] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:32.914218 13494 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:32.904170 13465 consensus_meta.cc:287] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.728s user 0.009s sys 0.000s
W20260809 15:52:32.910135 10922 consensus_meta.cc:287] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.729s user 0.011s sys 0.000s
I20260809 15:52:32.850682 13447 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:32.851055 13276 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:32.915881 13511 raft_consensus.cc:3032] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:32.863881 13360 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:32.966002 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 112461us
I20260809 15:52:32.907199 13322 consensus_queue.cc:1035] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260809 15:52:32.968268 13425 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:32.963425 10922 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:32.958699 13344 consensus_queue.cc:1046] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260809 15:52:32.972215 10893 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:32.906432 13326 consensus_queue.cc:1046] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.067s
I20260809 15:52:32.918207 13449 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:32.975265 13495 log.cc:924] Time spent T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.462s user 0.001s sys 0.000s
I20260809 15:52:32.939018 13233 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:32.978026 13490 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:32.978547 13424 raft_consensus.cc:2726] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 15:52:32.969894 13232 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:32.978919 13417 consensus_queue.cc:237] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:32.975705 13445 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:32.970315 13511 raft_consensus.cc:729] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:32.981015 13464 raft_consensus.cc:2726] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:32.983211 10893 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:32.875269 13333 consensus_queue.cc:1035] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260809 15:52:32.969673 13447 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:32.979583 13487 log.cc:924] Time spent T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.179s user 0.000s sys 0.001s
I20260809 15:52:32.947304 13465 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:32.990733 13416 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:33.040879 13424 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:33.043383 13424 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:33.044452 13513 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:33.044981 13513 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:33.045404 13424 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:33.046814 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 63767.9us
I20260809 15:52:33.047021 13513 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:33.049240 11013 raft_consensus.cc:2379] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 4: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
W20260809 15:52:33.049412 13507 log.cc:924] Time spent T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.091s user 0.001s sys 0.000s
W20260809 15:52:33.049723 13500 log.cc:924] Time spent T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.360s user 0.001s sys 0.000s
I20260809 15:52:33.051147 13360 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:33.057157 13477 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:33.059224 13465 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:33.060110 13435 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:33.061655 13499 log.cc:924] Time spent T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.223s user 0.004s sys 0.000s
I20260809 15:52:33.049573 13326 raft_consensus.cc:2726] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:33.062667 13312 consensus_queue.cc:237] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:33.047809 10893 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:33.063725 13427 raft_consensus.cc:3032] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:33.128885 10893 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:33.132776 13464 raft_consensus.cc:2726] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260809 15:52:33.129611 13207 consensus_meta.cc:287] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.947s user 0.010s sys 0.003s
I20260809 15:52:33.040469 13445 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:33.126791 13424 raft_consensus.cc:2726] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 15:52:33.124271 13427 raft_consensus.cc:729] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:33.141207 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:33.127894 13416 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:33.145116 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:33.004957 13490 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:33.151257 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:33.151477 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:33.152467 11636 raft_consensus.cc:2354] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4.
I20260809 15:52:33.124923 13494 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:33.153690 10811 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.531s user 0.011s sys 0.081s
W20260809 15:52:33.154351 10811 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.539s user 0.011s sys 0.081s
I20260809 15:52:33.154398 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
W20260809 15:52:33.144426 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 497146us
I20260809 15:52:33.152164 11009 raft_consensus.cc:2454] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
W20260809 15:52:33.157349 11010 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s user 0.001s sys 0.005s
W20260809 15:52:33.157712 11010 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.501s user 0.001s sys 0.005s
I20260809 15:52:33.134483 13326 consensus_queue.cc:1046] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260809 15:52:33.160847 11637 raft_consensus.cc:1261] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:33.161237 11636 raft_consensus.cc:1261] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 5 LEARNER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260809 15:52:33.163990 11606 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:33.163931 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:33.170934 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:33.140149 13207 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:33.176605 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:33.154136 13276 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:33.179977 10838 raft_consensus.cc:1261] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260809 15:52:33.183631 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:33.158639 13322 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:33.153904 13416 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:33.187731 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:33.188699 10930 raft_consensus.cc:2379] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 4: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:33.191076 10843 raft_consensus.cc:2379] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 2: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:33.170194 13382 raft_consensus.cc:3032] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:33.195837 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:33.196938 10930 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 6
I20260809 15:52:33.197424 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:33.211534 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:33.212306 10923 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:33.215744 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:33.216056 10924 raft_consensus.cc:1226] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260809 15:52:33.217919 11015 raft_consensus.cc:2354] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4.
I20260809 15:52:33.218642 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:33.200470 13382 raft_consensus.cc:729] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:33.160030 13427 consensus_queue.cc:260] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:33.221918 13427 raft_consensus.cc:3037] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:33.221522 13382 consensus_queue.cc:260] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:33.222668 13382 raft_consensus.cc:3037] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:33.223217 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:33.223284 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:33.225306 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:33.225687 13498 log.cc:924] Time spent T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.382s user 0.001s sys 0.002s
I20260809 15:52:33.226155 10843 raft_consensus.cc:1261] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 5. (term mismatch)
I20260809 15:52:33.226518 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:33.183140 13425 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:33.265779 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:33.266489 10847 raft_consensus.cc:2454] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:33.267479 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:33.268875 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:33.226523 10843 pending_rounds.cc:76] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536: Aborting all ops after (but not including) 4
I20260809 15:52:33.220096 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:33.136250 13511 consensus_queue.cc:260] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:33.272346 10842 raft_consensus.cc:2454] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:33.272450 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:33.274326 10927 raft_consensus.cc:2379] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 5: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:33.275607 10838 raft_consensus.cc:2379] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 4: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:33.276569 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:33.278901 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:33.280853 11013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50542 (request call id 2159) took 1069 ms. Trace:
I20260809 15:52:33.281358 10841 raft_consensus.cc:1261] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 15:52:33.270952 10843 pending_rounds.cc:97] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536: Aborting uncommitted CHANGE_CONFIG_OP operation due to leader change: 3.5
I20260809 15:52:33.281224 11013 rpcz_store.cc:276] 0809 15:52:32.196713 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:32.196915 (+ 202us) service_pool.cc:227] Handling call
0809 15:52:33.227205 (+1030290us) spinlock_profiling.cc:243] Waited 800 ms on lock 0x7b540040b354. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:33.266271 (+ 39066us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1840754432}
I20260809 15:52:33.213925 13276 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:33.289304 10843 raft_consensus.cc:2894] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Aborting config change with OpId 3.5: Aborted: Op aborted by new leader
I20260809 15:52:33.285967 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:33.293385 13490 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:33.294770 10843 raft_consensus.cc:2866] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: CHANGE_CONFIG_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:33.299734 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:33.300480 10839 raft_consensus.cc:2379] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 2: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:33.303546 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:33.227545 13424 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:33.199266 13322 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:33.413713 10844 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.705s user 0.007s sys 0.016s
I20260809 15:52:33.378041 10924 raft_consensus.cc:1261] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
W20260809 15:52:33.661211 10844 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.953s user 0.009s sys 0.031s
W20260809 15:52:33.382844 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 60986.4us
I20260809 15:52:33.358652 11016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 1952) took 1205 ms. Trace:
I20260809 15:52:33.336366 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:33.306792 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:33.663080 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:33.331542 11018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2144) took 1722 ms. Trace:
W20260809 15:52:33.358915 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 451964us
I20260809 15:52:33.663949 10848 raft_consensus.cc:3037] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:33.665858 10930 raft_consensus.cc:1261] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260809 15:52:33.336939 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:33.315440 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:33.664162 11018 rpcz_store.cc:276] 0809 15:52:31.609430 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:31.609584 (+ 154us) service_pool.cc:227] Handling call
0809 15:52:31.609825 (+ 241us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:32.406963 (+797138us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:33.331328 (+924365us) spinlock_profiling.cc:243] Waited 795 ms on lock 0x7b540004ddd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:33.331490 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1828831104}
I20260809 15:52:33.671844 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:33.662218 10893 leader_election.cc:398] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Vote denied by peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685) with higher term. Message: Invalid argument: T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4.
I20260809 15:52:33.675088 10893 leader_election.cc:401] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
I20260809 15:52:33.677958 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
W20260809 15:52:33.315857 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 102056us
I20260809 15:52:33.672644 11641 raft_consensus.cc:2454] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:52:33.662817 11016 rpcz_store.cc:276] 0809 15:52:32.153326 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:32.153457 (+ 131us) service_pool.cc:227] Handling call
0809 15:52:32.172063 (+ 18606us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:32.184874 (+ 12811us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:32.184902 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:32.193318 (+ 8416us) log.cc:842] Serialized 26 byte log entry
0809 15:52:32.379322 (+186004us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:32.379385 (+ 63us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:32.379479 (+ 94us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:33.224937 (+845458us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:52:33.315571 (+ 90634us) raft_consensus.cc:1671] finished
0809 15:52:33.315598 (+ 27us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:33.358585 (+ 42987us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":12501,"raft.queue_time_us":16253,"raft.run_cpu_time_us":96,"raft.run_wall_time_us":7080,"spinlock_wait_cycles":28998912,"thread_start_us":108012,"threads_started":1,"wal-append.queue_time_us":407384}
W20260809 15:52:33.737666 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 73188.4us
W20260809 15:52:33.739087 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 77054.3us
W20260809 15:52:33.377981 13299 consensus_meta.cc:287] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.740s user 0.011s sys 0.000s
I20260809 15:52:33.823318 10841 raft_consensus.cc:1226] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:33.768321 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:33.826015 10978 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.641s user 0.022s sys 0.011s
W20260809 15:52:33.611960 13520 log.cc:924] Time spent T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.096s user 0.000s sys 0.000s
I20260809 15:52:33.294806 13511 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
W20260809 15:52:33.417879 11603 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.551s user 0.002s sys 0.018s
W20260809 15:52:33.831233 10978 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.646s user 0.022s sys 0.012s
I20260809 15:52:33.679433 11606 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:33.513054 13431 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:33.336191 13509 raft_consensus.cc:2781] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:33.513060 13360 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:33.840574 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:33.558069 13519 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:33.557047 13510 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:33.514869 13494 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:33.652412 13521 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:52:33.859078 10926 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.633s user 0.002s sys 0.035s
I20260809 15:52:33.850984 10927 raft_consensus.cc:1226] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:33.480988 13465 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:33.850364 10847 raft_consensus.cc:1261] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260809 15:52:33.895756 10926 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.670s user 0.006s sys 0.035s
I20260809 15:52:33.553789 13516 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:33.666906 13478 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:33.644752 13207 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:33.667408 13317 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:33.655709 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:33.434094 13445 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:33.854267 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 669644us
I20260809 15:52:33.857338 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:33.434278 13354 raft_consensus.cc:686] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:33.727911 13416 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:33.722170 13470 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:52:33.853049 11603 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.058s user 0.006s sys 0.018s
I20260809 15:52:33.720400 13326 raft_consensus.cc:482] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:33.862134 10924 raft_consensus.cc:2454] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
W20260809 15:52:34.007395 13525 log.cc:924] Time spent T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.453s user 0.000s sys 0.001s
I20260809 15:52:33.850314 13473 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:34.008299 13529 log.cc:924] Time spent T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.417s user 0.000s sys 0.000s
I20260809 15:52:34.008122 13473 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:33.301756 13276 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:33.358062 13419 consensus_queue.cc:1035] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.500s
I20260809 15:52:33.280869 13425 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:33.850551 13484 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:33.939302 10840 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.714s user 0.042s sys 0.094s
W20260809 15:52:34.015061 10840 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.790s user 0.042s sys 0.094s
I20260809 15:52:33.676919 13424 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:33.934974 13517 log.cc:924] Time spent T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.733s user 0.000s sys 0.001s
I20260809 15:52:33.874794 13360 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:33.935292 13256 raft_consensus.cc:2726] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:33.874537 13518 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:33.839618 13299 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:33.896166 13530 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:33.931769 13527 raft_consensus.cc:2726] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685) with higher term. Message: Invalid argument: T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4.
I20260809 15:52:33.974756 10847 raft_consensus.cc:1226] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:33.976615 13445 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:33.862396 13431 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:34.015969 13207 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:34.038520 10838 raft_consensus.cc:2354] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6.
I20260809 15:52:34.009037 13416 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:52:34.043073 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 337168us
I20260809 15:52:34.049746 13419 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:33.940086 13494 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:34.056290 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 315173us
I20260809 15:52:34.029881 10921 raft_consensus.cc:2379] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 4: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:34.076421 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:34.164867 11016 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:34.076431 11010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2162) took 1508 ms. Trace:
I20260809 15:52:34.166309 11010 rpcz_store.cc:276] 0809 15:52:32.568147 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:32.568564 (+ 417us) service_pool.cc:227] Handling call
0809 15:52:32.596920 (+ 28356us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:32.604625 (+ 7705us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:32.604655 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:32.622830 (+ 18175us) log.cc:842] Serialized 26 byte log entry
0809 15:52:33.158791 (+535961us) raft_consensus.cc:1631] Marking committed up to 3
0809 15:52:33.158833 (+ 42us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:33.158912 (+ 79us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:34.040201 (+881289us) raft_consensus.cc:1671] finished
0809 15:52:34.040226 (+ 25us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:34.076332 (+ 36106us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3614,"raft.queue_time_us":34224,"raft.run_cpu_time_us":144,"raft.run_wall_time_us":8790,"thread_start_us":501631,"threads_started":1,"wal-append.queue_time_us":776945}
I20260809 15:52:33.939217 13465 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:33.973294 13317 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.170234 13317 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:33.939043 13510 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:33.877808 13519 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.170717 13465 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:33.970323 13478 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.112978 10809 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:34.172004 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 59115.2us
I20260809 15:52:34.172289 13510 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:34.173003 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:34.114707 13207 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:34.176265 13207 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:34.050655 13518 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.034422 13484 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:34.149354 13533 log.cc:924] Time spent T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.134s user 0.000s sys 0.000s
I20260809 15:52:33.939534 13521 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.007457 13326 raft_consensus.cc:504] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.171304 13465 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:34.182161 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:33.976212 13516 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.186664 13524 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:34.154422 13538 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:34.010264 13470 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.205966 10927 raft_consensus.cc:1261] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:52:34.176817 13207 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:34.203754 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:34.114173 13509 raft_consensus.cc:686] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
W20260809 15:52:34.192716 13352 consensus_meta.cc:287] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 1.363s user 0.011s sys 0.002s
I20260809 15:52:34.106454 13445 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:34.196463 13322 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:34.192587 11014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2163) took 1624 ms. Trace:
I20260809 15:52:34.316622 10926 raft_consensus.cc:2354] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4.
I20260809 15:52:34.155104 13416 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:34.204114 13417 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.612s user 0.004s sys 0.016s
I20260809 15:52:34.334439 10927 raft_consensus.cc:1261] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 15:52:34.077245 13530 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.325886 11014 rpcz_store.cc:276] 0809 15:52:32.568458 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:32.568666 (+ 208us) service_pool.cc:227] Handling call
0809 15:52:32.569009 (+ 343us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:32.623004 (+ 53995us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:32.623033 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:32.633070 (+ 10037us) log.cc:842] Serialized 26 byte log entry
0809 15:52:33.006468 (+373398us) raft_consensus.cc:1631] Marking committed up to 2
0809 15:52:33.006516 (+ 48us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:33.006594 (+ 78us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:34.192315 (+1185721us) raft_consensus.cc:1671] finished
0809 15:52:34.192343 (+ 28us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:34.192515 (+ 172us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":40536,"raft.run_cpu_time_us":124,"raft.run_wall_time_us":9266,"thread_start_us":281201,"threads_started":2,"wal-append.queue_time_us":498986}
I20260809 15:52:34.159107 13354 consensus_queue.cc:237] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.179025 13419 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:34.345942 13556 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:34.323815 13352 raft_consensus.cc:2726] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260809 15:52:34.352392 13508 log.cc:924] Time spent T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f: Append to log took a long time: real 1.508s user 0.012s sys 0.000s
I20260809 15:52:34.335005 13256 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:34.293948 13465 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:34.353559 13431 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:34.376667 10974 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.603s user 0.000s sys 0.020s
W20260809 15:52:34.377089 10974 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.702s user 0.000s sys 0.021s
I20260809 15:52:34.353737 13481 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:34.354410 13519 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:34.385246 13167 raft_consensus.cc:3032] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:34.385859 13167 raft_consensus.cc:729] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
W20260809 15:52:34.357178 10848 consensus_meta.cc:287] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.693s user 0.007s sys 0.000s
W20260809 15:52:34.338486 13417 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.784s user 0.005s sys 0.017s
I20260809 15:52:34.354413 11017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 1961) took 1206 ms. Trace:
I20260809 15:52:34.386900 10848 raft_consensus.cc:2454] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
W20260809 15:52:34.388743 13531 log.cc:924] Time spent T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.077s user 0.000s sys 0.001s
I20260809 15:52:34.389132 10848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2110) took 1179 ms. Trace:
I20260809 15:52:34.389002 13167 consensus_queue.cc:260] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.389500 10848 rpcz_store.cc:276] 0809 15:52:33.209372 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:33.285542 (+ 76170us) service_pool.cc:227] Handling call
0809 15:52:34.389076 (+1103534us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:34.347643 13470 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:34.361335 13484 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:34.390254 10844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 636) took 2231 ms. Trace:
I20260809 15:52:34.321595 13524 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.390676 13470 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:34.362967 10922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 676) took 1186 ms. Trace:
I20260809 15:52:34.323886 13549 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:34.337450 13322 raft_consensus.cc:2781] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:34.353085 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:34.177191 13527 consensus_queue.cc:1035] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.166s
W20260809 15:52:34.396575 13523 log.cc:924] Time spent T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.236s user 0.000s sys 0.000s
I20260809 15:52:34.388442 11017 rpcz_store.cc:276] 0809 15:52:33.147544 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:33.147659 (+ 115us) service_pool.cc:227] Handling call
0809 15:52:33.148465 (+ 806us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:33.151099 (+ 2634us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:33.151144 (+ 45us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:33.156770 (+ 5626us) log.cc:842] Serialized 26 byte log entry
0809 15:52:33.512600 (+355830us) raft_consensus.cc:1631] Marking committed up to 2
0809 15:52:33.512652 (+ 52us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:33.512735 (+ 83us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:34.354185 (+841450us) raft_consensus.cc:1671] finished
0809 15:52:34.354218 (+ 33us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:34.354352 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":2877,"raft.run_cpu_time_us":172,"raft.run_wall_time_us":8280,"thread_start_us":301212,"threads_started":1,"wal-append.queue_time_us":768960}
I20260809 15:52:34.370323 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:34.400913 10927 raft_consensus.cc:2454] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:34.385335 13519 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:34.356057 10843 raft_consensus.cc:1261] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 15:52:34.311103 13538 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:34.322942 13312 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:34.321910 13207 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:34.430439 13465 leader_election.cc:288] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:34.430177 13312 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.431217 13207 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:34.432317 13312 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:34.390654 10844 rpcz_store.cc:276] 0809 15:52:32.159128 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:32.159290 (+ 162us) service_pool.cc:227] Handling call
0809 15:52:32.159586 (+ 296us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:32.612251 (+452665us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:32.612280 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:32.617306 (+ 5026us) log.cc:842] Serialized 26 byte log entry
0809 15:52:33.661645 (+1044339us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:52:33.661680 (+ 35us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:33.661748 (+ 68us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:34.389988 (+728240us) raft_consensus.cc:1671] finished
0809 15:52:34.390016 (+ 28us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:34.390166 (+ 150us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":17444,"raft.queue_time_us":28074,"raft.run_cpu_time_us":157,"raft.run_wall_time_us":7587,"thread_start_us":1391721,"threads_started":2,"wal-append.queue_time_us":1462513}
I20260809 15:52:34.403213 10843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2106) took 1197 ms. Trace:
I20260809 15:52:34.434248 10843 rpcz_store.cc:276] 0809 15:52:33.205647 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:34.355465 (+1149818us) service_pool.cc:227] Handling call
0809 15:52:34.355761 (+ 296us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:34.402878 (+ 47117us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:34.403132 (+ 254us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:34.398883 10922 rpcz_store.cc:276] 0809 15:52:33.176708 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:33.179423 (+ 2715us) service_pool.cc:227] Handling call
0809 15:52:33.225527 (+ 46104us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:33.324069 (+ 98542us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:33.330960 (+ 6891us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:33.537612 (+206652us) log.cc:842] Serialized 26 byte log entry
0809 15:52:34.103473 (+565861us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:34.105980 (+ 2507us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:34.114955 (+ 8975us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:34.362742 (+247787us) raft_consensus.cc:1671] finished
0809 15:52:34.362768 (+ 26us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:34.362909 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":53857,"raft.queue_time_us":73501,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":36676,"thread_start_us":369113,"threads_started":1,"wal-append.queue_time_us":725523}
I20260809 15:52:34.436434 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:34.436830 10891 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:34.437531 10922 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:34.438181 13549 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:34.438803 13427 consensus_meta.cc:287] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 1.216s user 0.007s sys 0.003s
I20260809 15:52:34.396502 13470 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:34.384467 13299 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:34.363476 13556 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.440150 10922 raft_consensus.cc:595] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:34.457027 13207 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:34.478654 13207 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.480585 13207 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:34.481034 13207 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:34.381518 13256 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.461513 13521 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:34.467154 13218 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
W20260809 15:52:34.470747 11015 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50528 (request call id 1949) took 2320 ms (client timeout 1660 ms). Trace:
I20260809 15:52:34.428956 10921 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:34.397944 13322 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:34.478144 13516 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:34.383831 13360 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:34.428714 13276 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:34.473291 11636 raft_consensus.cc:2932] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:34.481472 13207 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:34.484038 11604 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.624s user 0.044s sys 0.093s
W20260809 15:52:34.502774 11015 rpcz_store.cc:269] 0809 15:52:32.150311 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:32.150740 (+ 429us) service_pool.cc:227] Handling call
0809 15:52:34.470484 (+2319744us) spinlock_profiling.cc:243] Waited 1.04 s on lock 0x7b540040b854. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:34.470642 (+ 158us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2394032512}
W20260809 15:52:34.533493 11604 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.674s user 0.044s sys 0.093s
I20260809 15:52:34.454010 13465 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:52:34.533892 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 680405us
I20260809 15:52:34.476624 13326 leader_election.cc:288] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:34.440291 13519 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.524466 13322 raft_consensus.cc:3037] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:34.543020 13424 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:34.543258 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:34.535185 13516 raft_consensus.cc:482] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:34.544768 13256 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:34.547103 13470 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:34.504345 13382 consensus_meta.cc:287] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 1.277s user 0.006s sys 0.006s
I20260809 15:52:34.498625 13494 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:34.548930 11018 raft_consensus.cc:1226] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:34.553915 13524 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:34.554162 13532 log.cc:924] Time spent T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.378s user 0.001s sys 0.000s
I20260809 15:52:34.567031 11018 raft_consensus.cc:1261] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch)
I20260809 15:52:34.525480 13218 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.359674 13509 consensus_queue.cc:237] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:34.556263 10838 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2111) took 1346 ms (client timeout 1645 ms). Trace:
W20260809 15:52:34.570694 10838 rpcz_store.cc:269] 0809 15:52:33.209649 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:33.278134 (+ 68485us) service_pool.cc:227] Handling call
0809 15:52:34.556038 (+1277904us) spinlock_profiling.cc:243] Waited 759 ms on lock 0x7b54000596d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:34.556195 (+ 157us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1744867072}
I20260809 15:52:34.542297 13465 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:34.557492 13426 consensus_queue.cc:260] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.559546 11015 raft_consensus.cc:2454] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:34.442059 13481 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.562456 13516 raft_consensus.cc:504] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.569955 13218 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:34.553998 13445 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:34.579633 13511 consensus_meta.cc:287] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.725s user 0.008s sys 0.001s
I20260809 15:52:34.544813 13544 consensus_queue.cc:1035] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.206s
I20260809 15:52:34.577636 13416 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:34.577468 13518 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:34.624027 10811 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.518s user 0.025s sys 0.035s
W20260809 15:52:34.628018 10811 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.522s user 0.025s sys 0.035s
I20260809 15:52:34.596060 13549 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:34.605858 13419 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:34.621414 13444 consensus_queue.cc:260] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.8, Last appended by leader: 8, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:34.645581 13419 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:34.641781 13538 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:34.726181 13481 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:34.704207 13416 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:34.762880 13444 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:34.720903 13530 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:34.864090 11606 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.004s user 0.021s sys 0.048s
W20260809 15:52:34.864478 11606 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.005s user 0.021s sys 0.048s
W20260809 15:52:34.865022 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 1.18563e+06us
W20260809 15:52:34.841745 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 139773us
I20260809 15:52:34.767750 13207 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:34.772133 13465 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:34.737520 13419 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:34.974766 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 108997us
I20260809 15:52:35.053382 10811 leader_election.cc:398] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Vote denied by peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783) with higher term. Message: Invalid argument: T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4.
I20260809 15:52:35.053964 10811 leader_election.cc:401] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Cancelling election due to peer responding with higher term
W20260809 15:52:34.982775 13559 log.cc:924] Time spent T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.282s user 0.000s sys 0.000s
W20260809 15:52:34.762533 13553 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.595s user 0.013s sys 0.051s
W20260809 15:52:35.072081 10809 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.880s user 0.013s sys 0.016s
W20260809 15:52:35.072572 10809 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.880s user 0.013s sys 0.016s
I20260809 15:52:34.765520 13519 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:35.073890 13553 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.906s user 0.013s sys 0.052s
W20260809 15:52:35.057905 13510 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.885s user 0.040s sys 0.041s
I20260809 15:52:34.985917 13556 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:35.084476 10892 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.793s user 0.022s sys 0.037s
W20260809 15:52:35.096658 10892 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.890s user 0.022s sys 0.038s
I20260809 15:52:34.867614 13516 leader_election.cc:288] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:35.094839 10893 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.914s user 0.020s sys 0.048s
W20260809 15:52:35.102851 10893 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.922s user 0.020s sys 0.048s
W20260809 15:52:35.103217 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 922894us
W20260809 15:52:35.087419 10808 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.768s user 0.013s sys 0.045s
W20260809 15:52:35.103996 10808 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.784s user 0.014s sys 0.046s
I20260809 15:52:35.016980 13478 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:35.020331 10921 raft_consensus.cc:2454] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:34.772519 13416 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:35.072371 13317 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:35.113822 10893 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:35.105537 10924 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.899s user 0.007s sys 0.056s
W20260809 15:52:35.137491 10924 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.931s user 0.008s sys 0.056s
I20260809 15:52:35.094288 13503 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
W20260809 15:52:35.097980 13510 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.925s user 0.040s sys 0.042s
I20260809 15:52:35.115142 13249 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:35.104092 13544 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:35.024140 13322 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:35.116003 11013 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.787s user 0.025s sys 0.036s
I20260809 15:52:35.077068 13422 consensus_queue.cc:1035] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260809 15:52:35.117733 13541 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:35.151501 11013 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.823s user 0.025s sys 0.040s
I20260809 15:52:35.152891 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:35.155404 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:35.155840 11639 raft_consensus.cc:2454] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:35.142558 13249 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:35.136065 10978 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.803s user 0.002s sys 0.010s
W20260809 15:52:35.158061 10978 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.830s user 0.003s sys 0.010s
I20260809 15:52:35.154026 13541 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:35.152359 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:35.161175 10930 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.985s user 0.005s sys 0.025s
I20260809 15:52:35.123145 13326 raft_consensus.cc:482] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:35.161521 11009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 474) took 1506 ms. Trace:
I20260809 15:52:35.165750 11009 rpcz_store.cc:276] 0809 15:52:33.654731 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:33.835581 (+180850us) service_pool.cc:227] Handling call
0809 15:52:33.835952 (+ 371us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:34.038304 (+202352us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:34.038343 (+ 39us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:34.039426 (+ 1083us) log.cc:842] Serialized 26 byte log entry
0809 15:52:34.575475 (+536049us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:34.575513 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:34.575585 (+ 72us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:35.161327 (+585742us) raft_consensus.cc:1671] finished
0809 15:52:35.161345 (+ 18us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:35.161464 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":137804,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":23002,"thread_start_us":661940,"threads_started":2,"wal-append.queue_time_us":1078798}
I20260809 15:52:35.157181 11013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 466) took 1861 ms. Trace:
W20260809 15:52:35.163056 10930 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.987s user 0.005s sys 0.025s
I20260809 15:52:35.168419 13563 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:35.169180 13546 raft_consensus.cc:2781] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:35.168956 13563 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:35.164170 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 110961us
I20260809 15:52:35.171378 13563 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:35.110769 13530 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.045s
I20260809 15:52:35.118005 13478 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:35.168078 11013 rpcz_store.cc:276] 0809 15:52:33.295931 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:33.297370 (+ 1439us) service_pool.cc:227] Handling call
0809 15:52:33.673136 (+375766us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:33.673365 (+ 229us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 15:52:34.075811 (+402446us) log.cc:842] Serialized 14 byte log entry
0809 15:52:35.156792 (+1080981us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:35.156852 (+ 60us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:35.156907 (+ 55us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:35.156986 (+ 79us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:35.157114 (+ 128us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":738,"prepare.queue_time_us":513750,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":123296,"raft.run_cpu_time_us":133,"raft.run_wall_time_us":4770,"thread_start_us":1331976,"threads_started":3,"wal-append.queue_time_us":528319}
I20260809 15:52:35.160985 13572 raft_consensus.cc:2726] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:35.180128 10978 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
W20260809 15:52:35.187706 10974 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.809s user 0.000s sys 0.013s
I20260809 15:52:35.194031 11017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2176) took 1183 ms. Trace:
I20260809 15:52:35.172736 13516 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:35.206715 13516 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:35.139505 13317 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:35.199555 11017 rpcz_store.cc:276] 0809 15:52:34.011305 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:34.482159 (+470854us) service_pool.cc:227] Handling call
0809 15:52:34.482446 (+ 287us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:34.501936 (+ 19490us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:34.501979 (+ 43us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:34.502033 (+ 54us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:52:34.502090 (+ 57us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:35.193748 (+691658us) spinlock_profiling.cc:243] Waited 19.2 ms on lock 0x7b540004d3d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:35.193807 (+ 59us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:35.193961 (+ 154us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":44251392}
I20260809 15:52:35.174856 13207 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:35.172366 13513 consensus_queue.cc:1035] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260809 15:52:35.161547 13544 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:35.186985 13425 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:35.191519 13465 raft_consensus.cc:2726] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:35.250509 10811 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
W20260809 15:52:35.254300 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 82073.3us
I20260809 15:52:35.178360 13326 raft_consensus.cc:504] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:35.239670 10891 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.735s user 0.001s sys 0.011s
W20260809 15:52:35.199615 10890 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.017s user 0.024s sys 0.053s
I20260809 15:52:35.142854 13503 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:35.239032 10978 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:35.255492 11637 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
W20260809 15:52:35.259675 10890 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.100s user 0.025s sys 0.054s
I20260809 15:52:35.195768 13572 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:35.249910 10974 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.871s user 0.000s sys 0.014s
I20260809 15:52:35.184065 13478 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:35.277521 11013 raft_consensus.cc:1226] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:35.277685 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:35.278501 11017 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
W20260809 15:52:35.268046 10891 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.764s user 0.002s sys 0.011s
I20260809 15:52:35.281275 11009 raft_consensus.cc:1261] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
W20260809 15:52:35.253113 13344 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.615s user 0.004s sys 0.007s
W20260809 15:52:35.282328 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 793117us
I20260809 15:52:35.249908 13553 raft_consensus.cc:2726] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: Vote denied by peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783) with higher term. Message: Invalid argument: T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4.
I20260809 15:52:35.286521 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:35.268795 13516 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:35.252099 13573 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:35.250092 13571 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:35.311077 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 1
W20260809 15:52:35.292865 13344 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.655s user 0.004s sys 0.007s
I20260809 15:52:35.247320 13207 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:35.255792 13422 consensus_queue.cc:1035] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.039s
I20260809 15:52:35.316545 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:52:35.324314 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
W20260809 15:52:35.320230 13528 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.772s user 0.051s sys 0.060s
I20260809 15:52:35.289618 13541 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:35.332194 11636 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.751s user 0.016s sys 0.023s
W20260809 15:52:35.332952 11636 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.752s user 0.016s sys 0.023s
I20260809 15:52:35.293025 13465 raft_consensus.cc:2781] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:35.317380 11017 raft_consensus.cc:2454] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:35.295123 13588 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:35.321732 10978 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.524s (SENT) [suppressed 6 similar messages]
W20260809 15:52:35.331432 13528 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.790s user 0.052s sys 0.061s
W20260809 15:52:35.333333 13561 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.686s user 0.005s sys 0.026s
W20260809 15:52:35.312486 13508 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.941s user 0.003s sys 0.015s
I20260809 15:52:35.314327 13575 raft_consensus.cc:2781] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:35.332161 13419 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:35.377254 13574 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
W20260809 15:52:35.356366 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 254747us
I20260809 15:52:35.357822 11636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 770) took 2019 ms. Trace:
I20260809 15:52:35.356595 13588 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:35.378073 11636 rpcz_store.cc:276] 0809 15:52:33.338332 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:33.338463 (+ 131us) service_pool.cc:227] Handling call
0809 15:52:34.472684 (+1134221us) spinlock_profiling.cc:243] Waited 2.25 ms on lock 0x7b5800014198. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f7966593851 00007f7966593702 00007f796f483b09 00007f796e3bf730 00007f796e48579b 00007f796e485821 00007f796e4492b0 00007f796e4347f9 00007f796e4342a2 00007f7967b1a684
0809 15:52:34.472917 (+ 233us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:34.473164 (+ 247us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 15:52:34.580681 (+107517us) log.cc:842] Serialized 14 byte log entry
0809 15:52:35.357484 (+776803us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:35.357531 (+ 47us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:35.357586 (+ 55us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:35.357656 (+ 70us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:35.357770 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":81,"raft.run_cpu_time_us":146,"raft.run_wall_time_us":11211,"spinlock_wait_cycles":5169536,"thread_start_us":776047,"threads_started":1}
I20260809 15:52:35.367389 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:35.379906 11017 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 8.
I20260809 15:52:35.282598 13572 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:35.368829 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:35.349372 13322 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:35.372752 13326 leader_election.cc:288] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:35.322072 13571 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:35.374938 13570 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:35.351677 13249 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:35.379981 13588 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:35.358711 10976 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.757s user 0.000s sys 0.012s
I20260809 15:52:35.322808 13573 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:35.758033 10976 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.316s user 0.010s sys 0.034s
I20260809 15:52:35.750032 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:35.746876 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:35.381970 11636 raft_consensus.cc:1226] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260809 15:52:35.759871 11017 raft_consensus.cc:2396] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260809 15:52:35.413481 11014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2172) took 1437 ms. Trace:
I20260809 15:52:35.380930 11642 raft_consensus.cc:1226] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260809 15:52:35.761550 11014 rpcz_store.cc:276] 0809 15:52:33.975833 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:34.356319 (+380486us) service_pool.cc:227] Handling call
0809 15:52:34.356599 (+ 280us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:34.436857 (+ 80258us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:34.436901 (+ 44us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:34.436951 (+ 50us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:52:34.436993 (+ 42us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:35.413169 (+976176us) spinlock_profiling.cc:243] Waited 80 ms on lock 0x7b540002d354. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:35.413229 (+ 60us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:35.413406 (+ 177us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":184035584}
I20260809 15:52:35.763531 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:35.764783 11014 raft_consensus.cc:2379] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 4: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
W20260809 15:52:35.356773 13561 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.718s user 0.005s sys 0.027s
I20260809 15:52:35.766901 13577 raft_consensus.cc:482] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:35.767717 13577 raft_consensus.cc:504] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:35.770270 13584 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:35.770499 13344 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:35.770836 13584 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:35.771407 13595 log.cc:924] Time spent T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.346s user 0.000s sys 0.000s
I20260809 15:52:35.771546 13419 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:35.772205 13422 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:35.772094 13419 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:35.772696 13577 leader_election.cc:288] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:35.772989 13421 raft_consensus.cc:482] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:35.772864 13422 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:35.773743 13584 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:35.774289 13419 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:35.757467 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 284230us
I20260809 15:52:35.774901 13419 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:35.773550 13421 raft_consensus.cc:504] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:35.475973 11012 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.822s user 0.000s sys 0.022s
W20260809 15:52:35.788983 11012 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 2.135s user 0.006s sys 0.031s
I20260809 15:52:35.776415 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:35.789935 11017 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:52:35.381129 11015 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:35.782578 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:35.387754 13508 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.016s user 0.003s sys 0.015s
I20260809 15:52:35.426728 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:35.366954 13541 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:35.355645 13465 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 15:52:35.468164 13444 consensus_meta.cc:287] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.587s user 0.005s sys 0.006s
W20260809 15:52:35.461964 13354 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.106s user 0.036s sys 0.039s
I20260809 15:52:35.813573 10811 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:35.796648 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:35.793042 11009 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.508s user 0.007s sys 0.012s
W20260809 15:52:35.842213 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 51103.2us
I20260809 15:52:35.761435 13207 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:35.843048 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:35.534600 13509 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.900s user 0.000s sys 0.019s
I20260809 15:52:35.508379 13503 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:35.626472 13544 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:35.802021 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:35.927740 10893 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.570s user 0.019s sys 0.020s
I20260809 15:52:35.711884 13317 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:35.382519 13575 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:35.430186 13478 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:35.588986 13556 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
W20260809 15:52:35.930747 11013 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.572s user 0.010s sys 0.030s
I20260809 15:52:35.766180 13326 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:35.932263 11010 raft_consensus.cc:1261] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 5 index: 8. Preceding OpId from leader: term: 6 index: 8. (term mismatch)
I20260809 15:52:35.927982 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:35.377941 13574 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:35.950385 13509 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.320s user 0.000s sys 0.024s
I20260809 15:52:35.797390 13530 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:35.789031 13577 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:35.368636 13553 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:36.047636 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:36.047902 13553 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.048921 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:35.973654 13326 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:35.798106 13583 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:35.936939 11009 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.652s user 0.008s sys 0.021s
I20260809 15:52:35.938347 10927 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
W20260809 15:52:35.944478 11018 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.615s user 0.040s sys 0.019s
W20260809 15:52:36.060319 11018 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.730s user 0.041s sys 0.019s
W20260809 15:52:35.944789 10890 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.675s user 0.023s sys 0.024s
W20260809 15:52:36.061034 10890 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.792s user 0.023s sys 0.024s
W20260809 15:52:35.945009 10893 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.588s user 0.020s sys 0.020s
I20260809 15:52:35.793345 13584 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:35.953791 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:35.953802 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.065730 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:35.763173 13570 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:35.941685 10810 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.701s user 0.026s sys 0.027s
I20260809 15:52:35.771108 13344 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:35.904073 13416 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:35.955229 10978 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.597s user 0.004s sys 0.028s
I20260809 15:52:35.775348 13419 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:35.845899 13354 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.490s user 0.039s sys 0.057s
I20260809 15:52:35.971035 11010 pending_rounds.cc:76] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa: Aborting all ops after (but not including) 7
I20260809 15:52:36.074827 11010 pending_rounds.cc:97] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa: Aborting uncommitted NO_OP operation due to leader change: 5.8
I20260809 15:52:35.873744 13541 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:35.841708 11017 raft_consensus.cc:2454] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 1.
I20260809 15:52:36.021762 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.026418 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:36.026960 13421 leader_election.cc:288] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:36.028793 13574 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:35.973119 13556 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:35.968462 13575 raft_consensus.cc:3037] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:36.029583 13594 raft_consensus.cc:2781] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:36.030609 13573 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:35.952802 13572 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:36.033161 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:35.990042 13528 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.602s user 0.015s sys 0.019s
W20260809 15:52:36.128928 11011 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.530s user 0.015s sys 0.022s
I20260809 15:52:36.040756 11012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 468) took 2737 ms. Trace:
I20260809 15:52:36.070830 13584 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:36.050094 13553 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:35.840284 13465 raft_consensus.cc:3037] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:36.064232 13544 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:36.069204 10810 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.828s user 0.026s sys 0.027s
W20260809 15:52:36.072093 10978 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.714s user 0.004s sys 0.028s
I20260809 15:52:36.072263 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:36.135522 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 813740us
I20260809 15:52:36.136123 10927 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:36.059228 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:35.972419 11013 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.614s user 0.012s sys 0.031s
I20260809 15:52:36.130532 11012 rpcz_store.cc:276] 0809 15:52:33.296690 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:33.297185 (+ 495us) service_pool.cc:227] Handling call
0809 15:52:33.336750 (+ 39565us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:33.396007 (+ 59257us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:33.396036 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:33.654034 (+257998us) log.cc:842] Serialized 26 byte log entry
0809 15:52:35.789471 (+2135437us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:35.789509 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:35.789569 (+ 60us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:36.034267 (+244698us) raft_consensus.cc:1671] finished
0809 15:52:36.034290 (+ 23us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:36.034434 (+ 144us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":13305,"raft.queue_time_us":27447,"raft.run_cpu_time_us":231,"raft.run_wall_time_us":32019,"thread_start_us":2134985,"threads_started":1,"wal-append.queue_time_us":2376923}
I20260809 15:52:36.138000 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.138837 11012 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 1.
I20260809 15:52:36.062244 13607 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:36.098016 11010 raft_consensus.cc:2866] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:36.097765 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.142632 13506 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:36.142894 10926 raft_consensus.cc:2454] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:52:36.142439 13607 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.143163 13506 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.143751 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.145207 10926 raft_consensus.cc:1261] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 15:52:36.145444 13607 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:36.098806 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:36.102667 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.147610 13490 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:36.148201 10978 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:36.148150 13490 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:36.148476 10923 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52758 (request call id 680) took 2944 ms (client timeout 2876 ms). Trace:
W20260809 15:52:36.149381 10923 rpcz_store.cc:269] 0809 15:52:33.203827 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:33.224816 (+ 20989us) service_pool.cc:227] Handling call
0809 15:52:36.074702 (+2849886us) spinlock_profiling.cc:243] Waited 3.05 ms on lock 0x7b58000d1498. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f7966593851 00007f7966593702 00007f796f483b09 00007f796e3bf730 00007f796e48579b 00007f796e485821 00007f796e4351b0 00007f796e434a94 00007f7967b1cc04 00007f7967b1cb92
0809 15:52:36.147948 (+ 73246us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":7004032}
I20260809 15:52:36.060547 13577 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:36.114562 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.065236 13422 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:36.126490 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:36.034811 11015 raft_consensus.cc:2396] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260809 15:52:36.060070 13530 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.116173 13611 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:36.045572 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:36.132169 11011 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.533s user 0.015s sys 0.022s
I20260809 15:52:36.103587 13571 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:36.113910 13354 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:36.133924 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:36.067219 13334 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:52:36.129065 13528 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.755s user 0.015s sys 0.019s
I20260809 15:52:36.062508 13583 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.151374 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.118083 10921 raft_consensus.cc:2454] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:36.152529 13570 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:36.151147 11017 raft_consensus.cc:2454] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:52:36.176314 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.130311 13594 raft_consensus.cc:686] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:36.179134 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.174674 13541 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:36.184526 10838 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260809 15:52:36.185165 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:36.197127 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 383605us
I20260809 15:52:36.200683 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.247949 10840 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:36.249130 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.155143 13207 raft_consensus.cc:2726] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 15:52:36.207492 11639 raft_consensus.cc:2454] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:36.208559 13426 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
W20260809 15:52:36.209702 10842 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2101) took 3005 ms (client timeout 1952 ms). Trace:
W20260809 15:52:36.254148 10842 rpcz_store.cc:269] 0809 15:52:33.204141 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:33.204255 (+ 114us) service_pool.cc:227] Handling call
0809 15:52:36.209453 (+3005198us) spinlock_profiling.cc:243] Waited 48.4 ms on lock 0x7b540022c754. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:36.209591 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":111252224}
I20260809 15:52:36.214003 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.181866 13326 leader_election.cc:288] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:36.242041 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:36.241972 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:36.245369 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.268227 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:36.182356 11011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 469) took 2885 ms. Trace:
I20260809 15:52:36.249938 10840 raft_consensus.cc:2454] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:36.247784 10848 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 8.
I20260809 15:52:36.256806 11642 raft_consensus.cc:1261] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:36.182031 13490 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:36.158299 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 216934us
I20260809 15:52:36.155320 11633 raft_consensus.cc:2454] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:36.261677 10927 raft_consensus.cc:1261] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260809 15:52:36.261744 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:36.262368 13598 log.cc:924] Time spent T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.146s user 0.001s sys 0.000s
I20260809 15:52:36.241534 13334 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.281013 13583 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:36.281133 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.281839 11017 raft_consensus.cc:2454] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
W20260809 15:52:36.264076 13612 log.cc:924] Time spent T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.110s user 0.000s sys 0.000s
I20260809 15:52:36.282560 13334 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:36.263285 13608 log.cc:924] Time spent T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.116s user 0.000s sys 0.000s
I20260809 15:52:36.264140 10893 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:36.266454 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.285019 13545 raft_consensus.cc:2726] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:36.285501 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.267172 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.286473 13545 consensus_queue.cc:1035] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:52:36.286741 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.286831 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.287444 10844 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:36.288025 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.288303 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.288470 10893 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:36.289990 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.290436 13545 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:36.152019 13544 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:36.290747 10848 raft_consensus.cc:2454] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:36.269153 11011 rpcz_store.cc:276] 0809 15:52:33.297028 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:33.297266 (+ 238us) service_pool.cc:227] Handling call
0809 15:52:35.177974 (+1880708us) spinlock_profiling.cc:243] Waited 3.61 ms on lock 0x7b5800027098. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f7966593851 00007f7966593702 00007f796f483b09 00007f796e3bf730 00007f796e48579b 00007f796e485821 00007f796e4492b0 00007f796e4347f9 00007f796e4342a2 00007f7967b1a684
0809 15:52:35.178231 (+ 257us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:35.178483 (+ 252us) raft_consensus.cc:1520] Early marking committed up to index 6
0809 15:52:35.598670 (+420187us) log.cc:842] Serialized 14 byte log entry
0809 15:52:36.181963 (+583293us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:36.182018 (+ 55us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:36.182072 (+ 54us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:36.182157 (+ 85us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:36.182292 (+ 135us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5211,"prepare.queue_time_us":871557,"prepare.run_cpu_time_us":31,"prepare.run_wall_time_us":32,"raft.queue_time_us":85,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":52845,"spinlock_wait_cycles":8303872,"thread_start_us":991349,"threads_started":2,"wal-append.queue_time_us":663807}
I20260809 15:52:36.291649 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.206005 13611 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.292205 10848 raft_consensus.cc:2454] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:36.245205 13527 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:36.293030 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.293594 10848 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:36.293509 13527 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.276479 10922 raft_consensus.cc:2454] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:36.295384 13344 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:36.296041 13435 raft_consensus.cc:3032] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:36.296342 13419 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:36.296581 13435 raft_consensus.cc:729] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:36.297067 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.269011 13444 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:36.299118 13444 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.301151 13444 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:36.301450 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.305502 10920 raft_consensus.cc:2379] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 4: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
W20260809 15:52:36.305979 10920 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 1946) took 3121 ms (client timeout 2118 ms). Trace:
W20260809 15:52:36.306424 10920 rpcz_store.cc:269] 0809 15:52:33.184251 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:33.186841 (+ 2590us) service_pool.cc:227] Handling call
0809 15:52:36.301243 (+3114402us) spinlock_profiling.cc:243] Waited 5.7 ms on lock 0x7b58000d1498. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f7966593851 00007f7966593702 00007f796f483b09 00007f796e3bf730 00007f796e48579b 00007f796e485821 00007f796e4351b0 00007f796e434a94 00007f7967b1cc04 00007f7967b1cb92
0809 15:52:36.305907 (+ 4664us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":13110144}
I20260809 15:52:36.307065 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.279791 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:36.308035 11640 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:36.308632 13577 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:36.309744 13546 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:36.281908 13473 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:36.309526 13594 consensus_queue.cc:237] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:36.310259 13546 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:36.297698 13435 consensus_queue.cc:260] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.311304 13435 raft_consensus.cc:3037] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:36.289567 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.303244 11604 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:36.268810 10847 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:36.278656 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.304072 11632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 585) took 4083 ms. Trace:
I20260809 15:52:36.304538 11018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 478) took 1015 ms. Trace:
I20260809 15:52:36.314517 10842 raft_consensus.cc:2454] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
W20260809 15:52:36.278982 13580 log.cc:924] Time spent T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.098s user 0.000s sys 0.000s
I20260809 15:52:36.278985 13556 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:36.270494 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.288698 10838 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:36.314579 11018 rpcz_store.cc:276] 0809 15:52:35.288658 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:35.288766 (+ 108us) service_pool.cc:227] Handling call
0809 15:52:35.289060 (+ 294us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:35.310037 (+ 20977us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:35.310102 (+ 65us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:35.316962 (+ 6860us) log.cc:842] Serialized 26 byte log entry
0809 15:52:36.063734 (+746772us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:36.063804 (+ 70us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:36.063873 (+ 69us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:36.289909 (+226036us) raft_consensus.cc:1671] finished
0809 15:52:36.289929 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:36.290063 (+ 134us) inbound_call.cc:173] Queueing success response
0809 15:52:36.293724 (+ 3661us) spinlock_profiling.cc:243] Waited 8.24 ms on lock 0x7b48000213d8. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796e39a7f1 00007f796e3918e7 00007f796e39177a 00007f796e48595f 00007f796e485627 00007f796e499b2c 00007f796e499ad8 00007f796e499a58 00007f796e499a13 00007f796e498cbc 00007f796d41abe6
Metrics: {"mutex_wait_us":20583,"raft.queue_time_us":22469,"raft.run_cpu_time_us":3037,"raft.run_wall_time_us":993559,"spinlock_wait_cycles":50726528,"thread_start_us":730794,"threads_started":1,"wal-append.queue_time_us":792114}
I20260809 15:52:36.314643 11604 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:36.322706 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.324846 10837 raft_consensus.cc:2454] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:36.326429 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.343483 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:36.342648 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.347755 11011 raft_consensus.cc:2454] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:36.339735 10842 raft_consensus.cc:1261] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:52:36.349315 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.326975 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.350991 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.351850 11013 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:52:36.352715 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.353753 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.355921 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.350980 11018 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:36.357242 11011 raft_consensus.cc:2396] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 4 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260809 15:52:36.358254 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:36.345966 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 57699.7us
I20260809 15:52:36.356266 11604 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:36.247107 13354 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.322414 13473 raft_consensus.cc:2781] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:36.367604 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.370270 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.349124 13299 raft_consensus.cc:2781] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:52:36.343961 13530 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:36.379876 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 202976us
I20260809 15:52:36.339267 13527 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:36.386989 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.404235 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.387333 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:36.404826 10840 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:36.404949 11018 raft_consensus.cc:3037] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:36.372120 13621 raft_consensus.cc:2781] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:52:36.406013 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.348924 13577 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:36.348732 13576 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:36.422497 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.350656 13444 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:36.406117 13621 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:36.387033 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.435542 11013 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:36.436625 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.404405 10842 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 8.
I20260809 15:52:36.412468 10841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2113) took 3202 ms. Trace:
I20260809 15:52:36.369880 13473 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:36.439589 13473 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:36.412271 13510 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:36.417649 13354 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:36.418147 13611 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:36.316206 11632 rpcz_store.cc:276] 0809 15:52:32.220995 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:32.223487 (+ 2492us) service_pool.cc:227] Handling call
0809 15:52:32.266788 (+ 43301us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:32.625798 (+359010us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:32.631694 (+ 5896us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:32.838637 (+206943us) log.cc:842] Serialized 26 byte log entry
0809 15:52:33.480194 (+641557us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:33.480245 (+ 51us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:36.303587 (+2823342us) spinlock_profiling.cc:243] Waited 329 ms on lock 0x7b54004f3dd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:36.303652 (+ 65us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:36.303724 (+ 72us) raft_consensus.cc:1671] finished
0809 15:52:36.303740 (+ 16us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:36.303995 (+ 255us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":146538,"raft.run_cpu_time_us":2081,"raft.run_wall_time_us":35339,"spinlock_wait_cycles":756252032,"thread_start_us":500761,"threads_started":1,"wal-append.queue_time_us":1156454,"wal-append.run_cpu_time_us":3110,"wal-append.run_wall_time_us":1428191}
I20260809 15:52:36.388382 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.388854 13299 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:36.419862 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.459591 11011 raft_consensus.cc:2396] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 15:52:36.420879 13584 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:36.421011 13544 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:36.421669 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.424225 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:36.429845 10929 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:36.426843 10809 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:36.434397 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.448122 11009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 1966) took 1164 ms. Trace:
I20260809 15:52:36.443693 10841 rpcz_store.cc:276] 0809 15:52:33.210239 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:33.863319 (+653080us) service_pool.cc:227] Handling call
0809 15:52:33.900650 (+ 37331us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:33.985178 (+ 84528us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:33.985217 (+ 39us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:36.218833 (+2233616us) spinlock_profiling.cc:243] Waited 9.2 ms on lock 0x7b2c000120e8. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7967ed6e11 00007f7967ee6442 00007f796952ec11 00007f79694f358a 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772
0809 15:52:36.219351 (+ 518us) log.cc:842] Serialized 26 byte log entry
0809 15:52:36.412100 (+192749us) raft_consensus.cc:1631] Marking committed up to 2
0809 15:52:36.412133 (+ 33us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:36.412214 (+ 81us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:36.412252 (+ 38us) raft_consensus.cc:1671] finished
0809 15:52:36.412262 (+ 10us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:36.412418 (+ 156us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":259762,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":348,"spinlock_wait_cycles":21151488,"thread_start_us":276378,"threads_started":2,"wal-append.queue_time_us":186666}
W20260809 15:52:36.434639 11606 connection.cc:232] RPC call timeout handler was delayed by -1.7255224200198427s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1519s for a response
W20260809 15:52:36.471509 11606 connection.cc:232] RPC call timeout handler was delayed by -1.6094224199769087s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1648s for a response
I20260809 15:52:36.470731 11009 rpcz_store.cc:276] 0809 15:52:35.283938 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:35.284050 (+ 112us) service_pool.cc:227] Handling call
0809 15:52:35.284301 (+ 251us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:35.284503 (+ 202us) raft_consensus.cc:1520] Early marking committed up to index 6
0809 15:52:36.059361 (+774858us) log.cc:842] Serialized 14 byte log entry
0809 15:52:36.447792 (+388431us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:36.447838 (+ 46us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:36.447915 (+ 77us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:36.447970 (+ 55us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:36.448072 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":787107,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":26309,"raft.run_cpu_time_us":95,"raft.run_wall_time_us":20710,"thread_start_us":1148155,"threads_started":3,"wal-append.queue_time_us":340417}
W20260809 15:52:36.471928 11606 connection.cc:232] RPC call timeout handler was delayed by -1.4150224199984223s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1864s for a response
W20260809 15:52:36.472244 11606 connection.cc:232] RPC call timeout handler was delayed by -1.1756224199780263s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.213s for a response
I20260809 15:52:36.472426 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:36.454190 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 186640us
I20260809 15:52:36.456818 10893 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:36.457135 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.402364 13539 consensus_queue.cc:1046] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.052s
I20260809 15:52:36.449144 13621 raft_consensus.cc:3037] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:36.433492 13527 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:36.442775 13510 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:36.474138 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 47400.3us
I20260809 15:52:36.474915 10893 leader_election.cc:398] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Vote denied by peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) with higher term. Message: Invalid argument: T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6.
I20260809 15:52:36.481819 10893 leader_election.cc:401] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260809 15:52:36.466570 11018 raft_consensus.cc:2454] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:36.473318 13321 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:36.473045 11009 raft_consensus.cc:2454] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:36.463421 13299 raft_consensus.cc:3037] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:36.485399 10891 leader_election.cc:302] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:36.478850 11606 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:36.434973 13426 consensus_queue.cc:1035] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
I20260809 15:52:36.495288 10978 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:36.484189 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.484220 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.499691 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.500399 11011 raft_consensus.cc:2379] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 3: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:36.501124 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.502209 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.484758 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.491638 10811 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:36.494484 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.504150 10926 raft_consensus.cc:2454] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:36.501811 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.496376 11606 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:36.449399 13444 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.451193 13619 consensus_queue.cc:1035] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.033s
I20260809 15:52:36.478502 13625 raft_consensus.cc:2781] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:36.478432 13607 raft_consensus.cc:2781] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:36.505519 11009 raft_consensus.cc:2454] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:36.515930 10809 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:36.519014 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.519774 11012 raft_consensus.cc:2454] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:36.520411 13513 raft_consensus.cc:2726] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:36.520709 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.520912 11606 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:36.521515 11012 raft_consensus.cc:2396] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 6 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4.
I20260809 15:52:36.522164 13482 raft_consensus.cc:3037] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:36.522321 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:36.525604 13630 log.cc:924] Time spent T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.072s user 0.000s sys 0.000s
I20260809 15:52:36.464202 13611 consensus_queue.cc:1035] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.029s
I20260809 15:52:36.517447 10811 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:36.515244 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.487054 13527 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.528363 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.529218 10926 raft_consensus.cc:2379] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 2: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:36.454135 13577 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.534029 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.495831 13528 raft_consensus.cc:2726] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:36.521035 13553 raft_consensus.cc:2726] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:36.435875 13576 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:36.530673 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.482779 13321 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.485960 13574 raft_consensus.cc:2726] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) with higher term. Message: Invalid argument: T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6.
I20260809 15:52:36.530164 10927 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:36.507498 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.514510 13646 raft_consensus.cc:2781] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:36.574595 13646 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:36.575381 13646 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:36.532670 10809 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:36.455646 13629 consensus_queue.cc:1035] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.062s
I20260809 15:52:36.532271 11606 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:36.519172 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.531044 11640 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:36.563640 11014 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:36.581388 13528 raft_consensus.cc:2726] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260809 15:52:36.581540 11640 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41478 (request call id 915) took 3369 ms (client timeout 1634 ms). Trace:
W20260809 15:52:36.582307 11640 rpcz_store.cc:269] 0809 15:52:33.212470 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:33.268372 (+ 55902us) service_pool.cc:227] Handling call
0809 15:52:36.279575 (+3011203us) spinlock_profiling.cc:243] Waited 5.26 ms on lock 0x7b5800014198. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f7966593851 00007f7966593702 00007f796f483b09 00007f796e3bf730 00007f796e48579b 00007f796e485821 00007f796e4351b0 00007f796e434a94 00007f7967b1cc04 00007f7967b1cb92
0809 15:52:36.581449 (+301874us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":12107136}
I20260809 15:52:36.564234 10811 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:36.506489 13573 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.054s
I20260809 15:52:36.538468 13541 raft_consensus.cc:2781] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:36.564891 11012 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260809 15:52:36.569583 10926 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 1.
W20260809 15:52:36.570166 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 83761us
I20260809 15:52:36.530129 13344 raft_consensus.cc:2781] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:36.584456 13541 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:36.584796 13344 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:36.504832 13509 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.063s
I20260809 15:52:36.517588 13625 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:36.585206 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.585616 13625 raft_consensus.cc:3037] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:36.586050 10926 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:36.587174 10811 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:36.584869 13541 raft_consensus.cc:3037] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
W20260809 15:52:36.579492 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 46924.3us
I20260809 15:52:36.588783 10893 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:36.563736 13519 raft_consensus.cc:2781] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:36.591118 10809 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:36.590713 11011 raft_consensus.cc:2454] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:36.600581 10809 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:36.579658 13477 raft_consensus.cc:2781] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:36.565912 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.585323 13344 raft_consensus.cc:3037] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:36.540832 13465 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.591034 13584 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:36.592303 10811 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:36.538937 13607 raft_consensus.cc:686] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:36.602533 13594 raft_consensus.cc:2781] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:36.602255 13561 raft_consensus.cc:2781] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:36.598647 13539 raft_consensus.cc:2781] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:36.591162 11009 raft_consensus.cc:1261] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260809 15:52:36.599085 13519 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:36.605454 13551 raft_consensus.cc:2781] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:36.611982 13619 raft_consensus.cc:2781] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:36.637086 11012 raft_consensus.cc:1226] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:36.605607 13477 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:36.642788 11013 raft_consensus.cc:2354] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3.
I20260809 15:52:36.613348 13527 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:36.642166 13551 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:36.640295 13539 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:36.639783 13561 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:36.638644 13584 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.646569 13477 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:36.657014 13628 consensus_queue.cc:1046] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:36.664039 11012 raft_consensus.cc:1226] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
W20260809 15:52:36.664753 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 132539us
W20260809 15:52:36.667299 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 75055.3us
I20260809 15:52:36.607317 13260 consensus_queue.cc:1035] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.035s
I20260809 15:52:36.676955 10811 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
W20260809 15:52:36.677364 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 192062us
I20260809 15:52:36.677978 13422 raft_consensus.cc:2781] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:36.678771 13422 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:36.679401 13422 raft_consensus.cc:3037] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:36.671383 10809 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:36.605533 13544 consensus_queue.cc:1035] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.033s
I20260809 15:52:36.639912 13594 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:36.641901 13519 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:36.702991 10811 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:36.703603 11012 raft_consensus.cc:1261] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:52:36.703055 10891 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:36.705698 11606 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:36.706945 13594 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:36.667393 13551 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:36.712075 11016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2171) took 2736 ms. Trace:
I20260809 15:52:36.666960 13571 raft_consensus.cc:2781] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:36.713065 10811 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:36.713126 10891 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:36.667969 13619 raft_consensus.cc:686] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
W20260809 15:52:36.722504 13648 log.cc:924] Time spent T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.123s user 0.000s sys 0.001s
I20260809 15:52:36.722787 13619 consensus_queue.cc:237] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.723402 13584 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:36.723529 13645 log.cc:924] Time spent T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.081s user 0.000s sys 0.000s
I20260809 15:52:36.668257 13561 raft_consensus.cc:3037] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:36.717497 11016 rpcz_store.cc:276] 0809 15:52:33.975536 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:34.174774 (+199238us) service_pool.cc:227] Handling call
0809 15:52:34.175142 (+ 368us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:34.484310 (+309168us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:34.484351 (+ 41us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:34.484401 (+ 50us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:52:34.484452 (+ 51us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:36.711850 (+2227398us) spinlock_profiling.cc:243] Waited 309 ms on lock 0x7b54001de054. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:36.711898 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:36.712016 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":710421248}
I20260809 15:52:36.667225 13539 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:36.708703 13545 raft_consensus.cc:2726] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:36.719255 13593 raft_consensus.cc:2781] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:36.714484 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.704403 13513 consensus_queue.cc:1046] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.030s
I20260809 15:52:36.721655 13571 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:36.704769 13546 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.733804 13473 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.757606 11010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 476) took 3090 ms. Trace:
I20260809 15:52:36.758114 11010 rpcz_store.cc:276] 0809 15:52:33.667337 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:34.172017 (+504680us) service_pool.cc:227] Handling call
0809 15:52:34.172495 (+ 478us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:36.142502 (+1970007us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:36.757324 (+614822us) spinlock_profiling.cc:243] Waited 1.71 s on lock 0x7b54002ef9d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:36.757530 (+ 206us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3923923200}
W20260809 15:52:36.751101 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 79811us
I20260809 15:52:36.717811 13650 raft_consensus.cc:2781] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:36.739264 13553 raft_consensus.cc:2781] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:36.748080 13545 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:36.747943 13527 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.015s
I20260809 15:52:36.752229 13574 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:36.760509 10809 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:36.746296 13593 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 15:52:36.772622 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 60204.6us
I20260809 15:52:36.792630 10809 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:36.757067 13635 log.cc:924] Time spent T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.234s user 0.000s sys 0.001s
I20260809 15:52:36.747689 13444 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:36.747632 13653 raft_consensus.cc:2781] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:36.755398 13571 raft_consensus.cc:3037] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:36.748734 13655 raft_consensus.cc:2726] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:36.796135 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:36.764509 13513 raft_consensus.cc:3032] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:36.762506 13473 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:36.762394 13289 raft_consensus.cc:3032] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:36.748240 13361 consensus_queue.cc:1035] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.038s
I20260809 15:52:36.769028 13553 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:36.771502 13576 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:36.800158 13653 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 15:52:36.806034 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 93040.1us
I20260809 15:52:36.742233 13607 consensus_queue.cc:237] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:36.794513 13593 raft_consensus.cc:3037] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:36.794963 13260 raft_consensus.cc:2781] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:36.770661 13650 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:36.828877 10811 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:36.803041 13465 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:36.786936 13545 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:36.863435 10929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 1947) took 3678 ms (client timeout 1634 ms). Trace:
I20260809 15:52:36.807389 13513 raft_consensus.cc:729] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:36.813081 13289 raft_consensus.cc:729] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:36.817355 13655 raft_consensus.cc:2726] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:36.841572 13506 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:36.794826 13574 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.844955 13260 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:52:36.834997 13643 log.cc:924] Time spent T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.147s user 0.000s sys 0.001s
I20260809 15:52:36.825640 13553 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:36.812884 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:36.858291 13641 log.cc:924] Time spent T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.353s user 0.002s sys 0.001s
W20260809 15:52:36.859988 13647 log.cc:924] Time spent T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.135s user 0.000s sys 0.000s
W20260809 15:52:36.860546 13435 consensus_meta.cc:287] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.549s user 0.008s sys 0.008s
I20260809 15:52:36.861611 13546 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:36.864022 10929 rpcz_store.cc:269] 0809 15:52:33.184588 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:33.184695 (+ 107us) service_pool.cc:227] Handling call
0809 15:52:36.862687 (+3677992us) spinlock_profiling.cc:243] Waited 3.06 s on lock 0x7b540034f154. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:36.863365 (+ 678us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":7042576128}
I20260809 15:52:36.865236 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:36.813683 13528 consensus_queue.cc:1035] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260809 15:52:36.835737 13653 raft_consensus.cc:3037] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
W20260809 15:52:36.858312 13575 consensus_meta.cc:287] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.732s user 0.009s sys 0.007s
I20260809 15:52:36.876089 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:36.865949 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 72474.3us
I20260809 15:52:36.842366 13656 raft_consensus.cc:2726] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:52:36.851261 13650 raft_consensus.cc:3037] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:36.895044 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:36.913245 11015 raft_consensus.cc:2454] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:36.879338 13260 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:36.872459 13482 raft_consensus.cc:2726] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:36.893744 13545 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:36.874568 13621 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:36.875819 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 166241us
W20260809 15:52:36.971310 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 48492us
I20260809 15:52:36.971913 11606 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:36.901012 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:36.897428 10922 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:36.905221 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.894450 13655 raft_consensus.cc:2726] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:36.903453 10924 raft_consensus.cc:2454] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:36.909861 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:36.864015 13551 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:36.891013 13423 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:36.914625 10893 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:36.919436 13477 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:36.920161 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:36.917043 10891 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:36.912614 10839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2108) took 3706 ms. Trace:
I20260809 15:52:36.900164 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:36.969995 13651 log.cc:924] Time spent T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.122s user 0.000s sys 0.001s
I20260809 15:52:36.971959 10809 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:36.973672 11636 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:36.910533 13289 consensus_queue.cc:260] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:36.913087 13513 consensus_queue.cc:260] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:36.985896 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:36.991114 11606 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:36.909873 13575 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:37.023427 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:37.081804 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 90744.5us
W20260809 15:52:37.082099 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 165125us
W20260809 15:52:37.081300 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 166750us
W20260809 15:52:37.079761 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 107887us
I20260809 15:52:37.084291 10893 leader_election.cc:302] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:37.077663 13510 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:37.090291 10926 raft_consensus.cc:2379] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 4: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
W20260809 15:52:37.093462 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 264261us
I20260809 15:52:37.085796 10839 rpcz_store.cc:276] 0809 15:52:33.208760 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:36.302716 (+3093956us) service_pool.cc:227] Handling call
0809 15:52:36.323773 (+ 21057us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:36.323955 (+ 182us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:36.323982 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:36.324023 (+ 41us) raft_consensus.cc:1631] Marking committed up to 3
0809 15:52:36.324065 (+ 42us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:36.324119 (+ 54us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:36.912396 (+588277us) spinlock_profiling.cc:243] Waited 20.7 ms on lock 0x7b540001de50. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:36.912541 (+ 145us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":47540096}
I20260809 15:52:37.099403 13661 raft_consensus.cc:2781] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:37.100009 13661 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:37.100518 13661 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
W20260809 15:52:37.053049 13660 log.cc:924] Time spent T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.166s user 0.000s sys 0.001s
I20260809 15:52:37.109557 10843 raft_consensus.cc:1226] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:37.108584 10811 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:36.989017 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:37.082515 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.103044 13243 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 15:52:36.990465 13361 raft_consensus.cc:2726] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:37.121340 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:37.123682 10978 leader_election.cc:334] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Timed out: RequestConsensusVote RPC to 127.9.84.2:32783 timed out after 1.817s (SENT)
I20260809 15:52:37.124011 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:37.194522 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:37.121731 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.110438 13545 raft_consensus.cc:2781] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:36.993821 13528 consensus_queue.cc:1035] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.126s
I20260809 15:52:37.186838 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:37.187544 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:37.232018 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:37.232647 10842 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:37.188180 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:37.233062 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:37.118048 13289 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:37.237023 13621 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:37.097657 13435 raft_consensus.cc:2781] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:37.123728 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:37.196321 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:37.193849 13655 raft_consensus.cc:2726] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:37.074610 13667 consensus_queue.cc:1035] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.203s
W20260809 15:52:37.198698 13625 consensus_meta.cc:287] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.613s user 0.002s sys 0.009s
I20260809 15:52:37.194298 10978 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:37.254204 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:37.223673 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:37.025506 13423 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:37.108582 13422 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:37.199244 13504 consensus_queue.cc:1046] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
I20260809 15:52:37.117040 13243 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:37.236749 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:37.187090 13574 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:37.252449 13435 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:37.255775 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:37.259347 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:37.263264 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 139593us
I20260809 15:52:37.115875 13656 consensus_queue.cc:1046] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.136s
I20260809 15:52:37.265525 13465 raft_consensus.cc:2726] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:37.265919 13477 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:37.258566 13625 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:37.252501 13299 consensus_meta.cc:287] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.741s user 0.012s sys 0.000s
I20260809 15:52:37.270807 10926 raft_consensus.cc:2379] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 4: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:37.252737 13354 raft_consensus.cc:2726] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:37.191907 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:37.278213 10978 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:37.276790 10839 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:37.280671 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:37.282701 13656 raft_consensus.cc:3032] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:37.274307 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.273116 13243 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:52:37.247995 13561 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:37.318390 13666 log.cc:924] Time spent T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.271s user 0.001s sys 0.000s
W20260809 15:52:37.348975 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 264781us
I20260809 15:52:37.281394 13435 raft_consensus.cc:3037] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:37.297068 13504 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:37.321615 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:37.349720 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:37.311312 10978 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:37.281646 13551 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:37.362427 10838 raft_consensus.cc:2379] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 4: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:37.276304 13423 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:37.364477 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 53247us
I20260809 15:52:37.365614 10978 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:37.328856 13646 consensus_meta.cc:287] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.753s user 0.005s sys 0.008s
I20260809 15:52:37.326416 13465 raft_consensus.cc:2781] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:37.322275 13656 raft_consensus.cc:729] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:37.356104 13575 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:37.330402 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.274772 13553 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:37.296721 13299 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:37.376734 10893 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:37.386808 11606 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:37.395717 11606 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:37.396729 10893 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:37.397855 10893 leader_election.cc:302] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:37.397120 11606 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:37.374986 13646 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:37.399782 11016 raft_consensus.cc:1226] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
W20260809 15:52:37.384918 13668 log.cc:924] Time spent T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.324s user 0.001s sys 0.000s
I20260809 15:52:37.330952 13593 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:37.403342 13646 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:37.375808 13506 raft_consensus.cc:2781] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:37.330019 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:37.413895 11606 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:37.372280 13465 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:37.409780 13506 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:37.415366 13646 raft_consensus.cc:2781] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:37.369676 13504 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:37.401590 13530 raft_consensus.cc:2726] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
W20260809 15:52:37.404963 13619 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.519s user 0.029s sys 0.051s
I20260809 15:52:37.395467 10922 raft_consensus.cc:2454] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
W20260809 15:52:37.422753 13556 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.588s user 0.041s sys 0.041s
I20260809 15:52:37.411150 13354 raft_consensus.cc:2726] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:37.426088 13465 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:37.431252 13646 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:37.442636 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:37.421697 13423 raft_consensus.cc:2781] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:37.408196 13663 raft_consensus.cc:2726] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:37.404492 13482 consensus_queue.cc:1035] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.032s
I20260809 15:52:37.484174 13482 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:37.484787 13482 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:37.485375 13561 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:37.452183 11639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41482 (request call id 603) took 1399 ms. Trace:
I20260809 15:52:37.488891 11639 rpcz_store.cc:276] 0809 15:52:36.052822 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:36.069728 (+ 16906us) service_pool.cc:227] Handling call
0809 15:52:37.451974 (+1382246us) spinlock_profiling.cc:243] Waited 61.1 ms on lock 0x7b54006f45d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:37.452109 (+ 135us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":140506496}
W20260809 15:52:37.458846 13539 consensus_meta.cc:287] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.719s user 0.012s sys 0.000s
I20260809 15:52:37.490375 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:37.491250 11639 raft_consensus.cc:2454] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:52:37.490433 13539 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:37.492430 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:37.493322 11639 raft_consensus.cc:2379] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 4: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:37.433506 13506 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
W20260809 15:52:37.442852 13571 consensus_meta.cc:287] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.627s user 0.002s sys 0.008s
I20260809 15:52:37.427287 13662 raft_consensus.cc:2781] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260809 15:52:37.448601 13619 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.568s user 0.029s sys 0.052s
I20260809 15:52:37.467001 13423 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:37.506633 13423 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:37.502903 13571 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:37.444244 11640 raft_consensus.cc:1226] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 2. Current term is 4. Ops: []
W20260809 15:52:37.514959 11637 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41488 (request call id 773) took 2362 ms (client timeout 1609 ms). Trace:
W20260809 15:52:37.515403 11637 rpcz_store.cc:269] 0809 15:52:35.152172 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:35.153938 (+ 1766us) service_pool.cc:227] Handling call
0809 15:52:37.514726 (+2360788us) spinlock_profiling.cc:243] Waited 99.4 ms on lock 0x7b54007dafd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:37.514866 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":228520192}
W20260809 15:52:37.515781 13594 consensus_meta.cc:287] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.790s user 0.009s sys 0.001s
I20260809 15:52:37.465029 11014 raft_consensus.cc:2354] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4.
I20260809 15:52:37.517274 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:37.516467 13594 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:37.474328 13299 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:37.477963 10927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52758 (request call id 697) took 1103 ms. Trace:
I20260809 15:52:37.478404 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:37.478974 11013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39978 (request call id 495) took 1086 ms. Trace:
I20260809 15:52:37.478504 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:37.481412 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:54216 (request call id 1537) took 4410 ms. Trace:
I20260809 15:52:37.496161 10842 raft_consensus.cc:1799] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Handling vote request from an unknown peer d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:37.497154 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:37.481307 13593 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:37.605794 10839 raft_consensus.cc:2379] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 3: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:37.606305 10842 raft_consensus.cc:2354] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7.
W20260809 15:52:37.463588 13556 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.632s user 0.042s sys 0.042s
I20260809 15:52:37.508661 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:37.468595 13646 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:37.417011 13656 consensus_queue.cc:260] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:37.517889 11014 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:37.518901 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:37.469573 13676 raft_consensus.cc:2726] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:37.520373 10924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52758 (request call id 690) took 1202 ms. Trace:
I20260809 15:52:37.521549 10809 leader_election.cc:398] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Vote denied by peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223) with higher term. Message: Invalid argument: T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4.
I20260809 15:52:37.615622 11018 raft_consensus.cc:2379] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 4: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
W20260809 15:52:37.615986 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 94607.2us
I20260809 15:52:37.616189 11018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50528 (request call id 1982) took 1226 ms. Trace:
I20260809 15:52:37.616596 11018 rpcz_store.cc:276] 0809 15:52:36.390132 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:36.482508 (+ 92376us) service_pool.cc:227] Handling call
0809 15:52:37.518766 (+1036258us) spinlock_profiling.cc:243] Waited 1.37 ms on lock 0x7b5800027098. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f7966593851 00007f7966593702 00007f796f483b09 00007f796e3bf730 00007f796e48579b 00007f796e485821 00007f796e4351b0 00007f796e434a94 00007f7967b1cc04 00007f7967b1cb92
0809 15:52:37.616121 (+ 97355us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3158784}
I20260809 15:52:37.461380 13422 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:37.625048 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
W20260809 15:52:37.609994 13519 consensus_meta.cc:287] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.895s user 0.007s sys 0.006s
I20260809 15:52:37.499698 13625 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:37.524789 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:37.635388 11640 raft_consensus.cc:2379] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 2: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:37.501825 13539 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:37.637146 13553 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:37.542416 10927 rpcz_store.cc:276] 0809 15:52:36.374251 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:36.425031 (+ 50780us) service_pool.cc:227] Handling call
0809 15:52:37.477782 (+1052751us) spinlock_profiling.cc:243] Waited 59.3 ms on lock 0x7b54000a91d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:37.477894 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":136422144}
I20260809 15:52:37.637297 13519 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:37.645903 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.550827 11013 rpcz_store.cc:276] 0809 15:52:36.392495 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:36.456911 (+ 64416us) service_pool.cc:227] Handling call
0809 15:52:37.478771 (+1021860us) spinlock_profiling.cc:243] Waited 168 ms on lock 0x7b540004bfd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:37.478906 (+ 135us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":387277696}
I20260809 15:52:37.648393 13519 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:37.648523 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:37.649363 11013 raft_consensus.cc:3037] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
W20260809 15:52:37.650554 13581 log.cc:924] Time spent T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 2.173s user 0.008s sys 0.000s
I20260809 15:52:37.650992 11017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2185) took 1307 ms. Trace:
I20260809 15:52:37.651492 11017 rpcz_store.cc:276] 0809 15:52:36.343800 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:36.354336 (+ 10536us) service_pool.cc:227] Handling call
0809 15:52:36.372597 (+ 18261us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:36.577097 (+204500us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:36.577147 (+ 50us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:36.577200 (+ 53us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:36.577242 (+ 42us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:37.650749 (+1073507us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b54001ddb54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:37.650816 (+ 67us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:37.650937 (+ 121us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":469933952}
I20260809 15:52:37.653374 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:37.652993 13423 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:37.552942 10931 rpcz_store.cc:276] 0809 15:52:33.070567 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:33.070703 (+ 136us) service_pool.cc:227] Handling call
0809 15:52:37.480646 (+4409943us) spinlock_profiling.cc:243] Waited 3.4 s on lock 0x7b540034f154. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478af0 00007f79694fbf2a 00007f796e438c6e 00007f7967b2f807 00007f7967b2f792 00007f7967b2f6c1 00007f7967b2f63c 00007f7967b2e8f2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:37.481322 (+ 676us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":7817396480}
I20260809 15:52:37.655651 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.656916 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:37.657544 10931 raft_consensus.cc:2362] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4. Re-sending same reply.
I20260809 15:52:37.604422 11632 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:37.568297 13482 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:37.663600 13243 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:37.665037 11604 leader_election.cc:398] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Vote denied by peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223) with higher term. Message: Invalid argument: T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3.
I20260809 15:52:37.665340 11639 raft_consensus.cc:2379] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 9: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:37.570258 13321 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:37.571511 10921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 688) took 1298 ms. Trace:
W20260809 15:52:37.572427 13344 consensus_meta.cc:287] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.956s user 0.012s sys 0.001s
W20260809 15:52:37.574703 13653 consensus_meta.cc:287] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.665s user 0.010s sys 0.001s
I20260809 15:52:37.579316 13504 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:37.587328 11012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 497) took 1194 ms. Trace:
I20260809 15:52:37.592314 13571 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:37.540711 13260 consensus_meta.cc:287] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.567s user 0.009s sys 0.001s
I20260809 15:52:37.609014 13424 raft_consensus.cc:979] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:37.617086 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:37.615567 10924 rpcz_store.cc:276] 0809 15:52:36.317814 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:36.317957 (+ 143us) service_pool.cc:227] Handling call
0809 15:52:37.520220 (+1202263us) spinlock_profiling.cc:243] Waited 553 ms on lock 0x7b540024c7d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:37.520320 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1272852352}
I20260809 15:52:37.618918 10809 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:37.501765 13662 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
W20260809 15:52:37.529501 13673 log.cc:924] Time spent T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.246s user 0.001s sys 0.001s
I20260809 15:52:37.619009 10891 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:37.594499 10922 raft_consensus.cc:3037] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:37.628757 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:37.636619 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:37.636988 13541 consensus_meta.cc:287] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 1.028s user 0.004s sys 0.008s
I20260809 15:52:37.637087 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:37.615777 13435 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:37.508456 13661 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:37.663450 10930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 683) took 4357 ms. Trace:
I20260809 15:52:37.665383 11604 leader_election.cc:401] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Cancelling election due to peer responding with higher term
I20260809 15:52:37.747881 11017 raft_consensus.cc:2454] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:52:37.748242 11012 rpcz_store.cc:276] 0809 15:52:36.423040 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:36.707533 (+284493us) service_pool.cc:227] Handling call
0809 15:52:36.770877 (+ 63344us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:36.772266 (+ 1389us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:36.772289 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:36.772331 (+ 42us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:36.772373 (+ 42us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:36.772435 (+ 62us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:37.587126 (+814691us) spinlock_profiling.cc:243] Waited 57.9 ms on lock 0x7b54001def50. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:37.587257 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":133090048}
I20260809 15:52:37.749281 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:37.747357 10930 rpcz_store.cc:276] 0809 15:52:33.306202 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:33.842816 (+536614us) service_pool.cc:227] Handling call
0809 15:52:33.851984 (+ 9168us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:34.175561 (+323577us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:34.175603 (+ 42us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:34.176150 (+ 547us) log.cc:842] Serialized 26 byte log entry
0809 15:52:35.168727 (+992577us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:35.168857 (+ 130us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:35.171169 (+ 2312us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:37.537119 (+2365950us) spinlock_profiling.cc:243] Waited 2.24 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:52:37.659077 (+121958us) raft_consensus.cc:1671] finished
0809 15:52:37.659096 (+ 19us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:37.663330 (+ 4234us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":505895,"raft.run_cpu_time_us":189,"raft.run_wall_time_us":19555,"spinlock_wait_cycles":5143808,"thread_start_us":1304205,"threads_started":2,"wal-append.queue_time_us":1137554}
W20260809 15:52:37.753429 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 134582us
I20260809 15:52:37.660902 10927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 1966) took 1005 ms. Trace:
I20260809 15:52:37.756065 10927 rpcz_store.cc:276] 0809 15:52:36.655388 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.645598 (+990210us) service_pool.cc:227] Handling call
0809 15:52:37.660826 (+ 15228us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:37.673071 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.765580 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 1970) took 1048 ms. Trace:
I20260809 15:52:37.766682 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:37.766764 10931 rpcz_store.cc:276] 0809 15:52:36.717597 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.672792 (+955195us) service_pool.cc:227] Handling call
0809 15:52:37.765515 (+ 92723us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:37.767804 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
W20260809 15:52:37.765496 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 146555us
I20260809 15:52:37.770320 13571 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
W20260809 15:52:37.771159 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 106353us
I20260809 15:52:37.765059 13321 raft_consensus.cc:1067] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:37.775257 11604 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:37.750133 11017 raft_consensus.cc:2454] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:37.776815 10809 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:37.777880 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.775230 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.778937 10930 raft_consensus.cc:2454] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:37.779537 10930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 1962) took 1139 ms. Trace:
I20260809 15:52:37.779892 10930 rpcz_store.cc:276] 0809 15:52:36.640741 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.774063 (+1133322us) service_pool.cc:227] Handling call
0809 15:52:37.779475 (+ 5412us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:37.781268 13299 raft_consensus.cc:2781] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 15:52:37.782163 13556 raft_consensus.cc:3037] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 3
W20260809 15:52:37.695392 13650 consensus_meta.cc:287] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.724s user 0.008s sys 0.007s
I20260809 15:52:37.784602 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.784992 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:37.785220 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.786150 10844 raft_consensus.cc:2379] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 4: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:37.788342 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:37.786335 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.789754 10924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 1964) took 1146 ms. Trace:
I20260809 15:52:37.753126 13662 raft_consensus.cc:3037] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:37.790247 10924 rpcz_store.cc:276] 0809 15:52:36.643367 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.765919 (+1122552us) service_pool.cc:227] Handling call
0809 15:52:37.789688 (+ 23769us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:37.787420 10930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 1971) took 1051 ms. Trace:
I20260809 15:52:37.790738 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:37.684826 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:37.791658 11014 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:37.775892 13424 raft_consensus.cc:2781] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:37.794556 10931 raft_consensus.cc:2454] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:37.697364 13594 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:37.786712 13504 raft_consensus.cc:2781] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:37.782732 13465 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:37.792479 10930 rpcz_store.cc:276] 0809 15:52:36.736539 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.780602 (+1044063us) service_pool.cc:227] Handling call
0809 15:52:37.787354 (+ 6752us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:52:37.803778 11635 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.617s user 0.013s sys 0.035s
W20260809 15:52:37.810993 11635 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.629s user 0.013s sys 0.035s
I20260809 15:52:37.803215 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.804544 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.812923 10931 raft_consensus.cc:2454] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:37.813613 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 1961) took 1177 ms. Trace:
I20260809 15:52:37.814093 10931 rpcz_store.cc:276] 0809 15:52:36.636558 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.804268 (+1167710us) service_pool.cc:227] Handling call
0809 15:52:37.813549 (+ 9281us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:52:37.808642 10840 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.611s user 0.000s sys 0.048s
I20260809 15:52:37.786533 13299 raft_consensus.cc:686] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:37.752375 13344 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:37.815227 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.766112 13260 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:37.815552 10840 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.618s user 0.000s sys 0.049s
I20260809 15:52:37.751996 13506 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:37.765234 13541 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:37.823773 11012 raft_consensus.cc:1226] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:37.815321 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.823566 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.774478 13653 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:37.823012 10924 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:37.803524 13424 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:37.812706 13423 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:37.835330 13243 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:37.836015 10929 raft_consensus.cc:2379] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 4: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:37.836140 13344 leader_election.cc:288] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:37.847834 10837 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.599s user 0.015s sys 0.026s
I20260809 15:52:37.837205 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.810491 13504 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:37.838575 11013 raft_consensus.cc:2454] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
W20260809 15:52:37.852813 11009 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.825s user 0.012s sys 0.045s
W20260809 15:52:37.853359 11009 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.858s user 0.012s sys 0.045s
I20260809 15:52:37.853443 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:37.856513 11018 raft_consensus.cc:1226] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:37.857260 10930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 1960) took 1240 ms. Trace:
I20260809 15:52:37.857648 10930 rpcz_store.cc:276] 0809 15:52:36.617029 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.815248 (+1198219us) service_pool.cc:227] Handling call
0809 15:52:37.857198 (+ 41950us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:37.858695 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.859438 10930 raft_consensus.cc:2454] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:37.860687 10809 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:37.838771 10924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 1967) took 1171 ms. Trace:
I20260809 15:52:37.862131 10924 rpcz_store.cc:276] 0809 15:52:36.667338 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.791060 (+1123722us) service_pool.cc:227] Handling call
0809 15:52:37.838716 (+ 47656us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:37.844185 13424 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:37.842214 13541 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:37.865731 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:37.865782 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:37.840194 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.789515 13650 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:37.872023 10847 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.628s user 0.030s sys 0.037s
W20260809 15:52:37.872972 10847 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.629s user 0.031s sys 0.037s
I20260809 15:52:37.841640 13594 consensus_queue.cc:1046] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.026s
I20260809 15:52:37.840243 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:37.830478 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:37.848716 10837 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.602s user 0.016s sys 0.026s
W20260809 15:52:37.852100 10848 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.582s user 0.018s sys 0.025s
W20260809 15:52:37.878491 10848 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.608s user 0.018s sys 0.025s
I20260809 15:52:37.867789 13519 raft_consensus.cc:2781] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:37.856108 13504 raft_consensus.cc:3037] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:37.869517 10930 raft_consensus.cc:3037] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:37.870239 10927 raft_consensus.cc:2454] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:37.839362 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 1969) took 1145 ms. Trace:
W20260809 15:52:37.874176 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 508593us
I20260809 15:52:37.889796 10931 rpcz_store.cc:276] 0809 15:52:36.694434 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.815073 (+1120639us) service_pool.cc:227] Handling call
0809 15:52:37.839285 (+ 24212us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:52:37.875979 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 478178us
I20260809 15:52:37.868786 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:37.891546 11013 raft_consensus.cc:3037] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:37.765316 10921 rpcz_store.cc:276] 0809 15:52:36.273381 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:36.287069 (+ 13688us) service_pool.cc:227] Handling call
0809 15:52:36.303932 (+ 16863us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:36.304111 (+ 179us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:36.304137 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:36.304179 (+ 42us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:36.304214 (+ 35us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:36.304263 (+ 49us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:37.571199 (+1266936us) spinlock_profiling.cc:243] Waited 16.6 ms on lock 0x7b540025bcd0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:37.571403 (+ 204us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":38078720}
I20260809 15:52:37.874591 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
W20260809 15:52:37.846204 11015 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.767s user 0.024s sys 0.044s
W20260809 15:52:37.921782 11015 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.843s user 0.024s sys 0.044s
I20260809 15:52:37.922437 10924 raft_consensus.cc:2379] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 3: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:37.924948 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:37.925810 11636 raft_consensus.cc:2454] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
W20260809 15:52:37.888386 13631 log.cc:924] Time spent T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 1.253s user 0.000s sys 0.003s
I20260809 15:52:37.934036 11013 raft_consensus.cc:2454] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:37.990445 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:37.992349 10891 leader_election.cc:302] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:37.944337 10810 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.726s user 0.000s sys 0.036s
I20260809 15:52:37.979699 10978 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:37.994349 10810 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.806s user 0.000s sys 0.037s
I20260809 15:52:37.998973 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:38.000314 10811 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:38.001309 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.905467 10927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 1972) took 1152 ms. Trace:
I20260809 15:52:38.004814 10978 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:38.006803 10978 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:38.006071 10927 rpcz_store.cc:276] 0809 15:52:36.752774 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.834489 (+1081715us) service_pool.cc:227] Handling call
0809 15:52:37.905393 (+ 70904us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:52:37.920083 13417 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.505s user 0.004s sys 0.013s
I20260809 15:52:38.018472 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:38.002027 10931 raft_consensus.cc:2454] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:38.019054 10927 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:38.019482 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 1984) took 1158 ms. Trace:
I20260809 15:52:38.020309 10931 rpcz_store.cc:276] 0809 15:52:36.861327 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.001065 (+1139738us) service_pool.cc:227] Handling call
0809 15:52:38.019380 (+ 18315us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:37.923710 13435 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:38.021521 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:37.923512 13519 raft_consensus.cc:686] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
W20260809 15:52:38.030239 10923 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.554s user 0.025s sys 0.025s
W20260809 15:52:38.030853 10923 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.555s user 0.025s sys 0.025s
I20260809 15:52:38.009613 10893 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:38.036844 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:37.927728 13679 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:38.043040 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:38.018831 13417 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.607s user 0.005s sys 0.013s
I20260809 15:52:38.008253 13640 raft_consensus.cc:2781] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:38.012852 13423 raft_consensus.cc:2781] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:38.047643 10838 raft_consensus.cc:1261] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:38.019918 13661 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:38.024171 13561 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.537s user 0.008s sys 0.007s
I20260809 15:52:38.061326 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:37.881582 13299 consensus_queue.cc:237] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:38.078748 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:38.081483 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:38.056857 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.083052 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:38.083400 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:38.061067 13561 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.574s user 0.008s sys 0.007s
I20260809 15:52:38.059988 13417 raft_consensus.cc:2781] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:52:38.047277 13640 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:38.084782 10930 raft_consensus.cc:2454] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:38.085292 13640 raft_consensus.cc:3037] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:38.085911 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:38.086164 10811 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:38.085446 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:38.024909 13688 raft_consensus.cc:2726] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 15:52:38.084717 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.091361 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:38.049060 13465 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:38.093688 13465 raft_consensus.cc:2781] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:38.094069 13465 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:38.094465 13465 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:38.094106 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:38.099403 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 1956) took 1496 ms. Trace:
I20260809 15:52:38.100059 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.099822 11015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 507) took 1128 ms. Trace:
I20260809 15:52:38.101339 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.100868 11015 rpcz_store.cc:276] 0809 15:52:36.971878 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:36.972036 (+ 158us) service_pool.cc:227] Handling call
0809 15:52:36.972340 (+ 304us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:36.972806 (+ 466us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:36.972834 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:37.078391 (+105557us) log.cc:842] Serialized 26 byte log entry
0809 15:52:37.925120 (+846729us) raft_consensus.cc:1631] Marking committed up to 7
0809 15:52:37.925161 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:37.925225 (+ 64us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:38.099615 (+174390us) raft_consensus.cc:1671] finished
0809 15:52:38.099636 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:38.099759 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":708,"raft.run_cpu_time_us":125,"raft.run_wall_time_us":3377,"thread_start_us":846203,"threads_started":1,"wal-append.queue_time_us":1019512}
I20260809 15:52:38.102938 10927 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:38.103240 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:38.103811 11015 raft_consensus.cc:3037] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:38.103969 10811 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:38.104763 13688 raft_consensus.cc:2781] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:38.108579 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:38.110260 13688 raft_consensus.cc:686] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:38.110558 13519 consensus_queue.cc:237] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:38.111249 13688 consensus_queue.cc:237] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:38.099813 10931 rpcz_store.cc:276] 0809 15:52:36.603166 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.046923 (+1443757us) service_pool.cc:227] Handling call
0809 15:52:38.099341 (+ 52418us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.074064 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:38.091179 13553 raft_consensus.cc:2781] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:38.117183 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:38.115734 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:38.118719 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.086212 13417 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:38.131683 10931 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:38.132485 11636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41478 (request call id 934) took 1019 ms. Trace:
I20260809 15:52:38.132900 11636 rpcz_store.cc:276] 0809 15:52:37.112915 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.106219 (+993304us) service_pool.cc:227] Handling call
0809 15:52:38.132424 (+ 26205us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.132144 10927 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:38.047560 13679 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:38.122669 13577 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:38.123135 13326 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:38.125801 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.127142 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.128162 13611 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:38.073968 13260 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:38.073679 13506 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:38.101734 13556 raft_consensus.cc:2726] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election lost for term 2. Reason: Vote denied by peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223) with higher term. Message: Invalid argument: T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3.
I20260809 15:52:38.119740 11009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 505) took 1223 ms. Trace:
I20260809 15:52:38.091846 13423 raft_consensus.cc:686] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:38.136521 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
W20260809 15:52:38.140161 13646 consensus_meta.cc:287] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.511s user 0.014s sys 0.004s
I20260809 15:52:38.164179 13646 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:38.165772 11017 raft_consensus.cc:1226] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:38.160316 11640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41478 (request call id 924) took 1060 ms. Trace:
I20260809 15:52:38.166345 11640 rpcz_store.cc:276] 0809 15:52:37.099749 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.126851 (+1027102us) service_pool.cc:227] Handling call
0809 15:52:38.160242 (+ 33391us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.167557 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.168524 11640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41478 (request call id 933) took 1064 ms. Trace:
I20260809 15:52:38.169715 11640 rpcz_store.cc:276] 0809 15:52:37.103641 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.167230 (+1063589us) service_pool.cc:227] Handling call
0809 15:52:38.168462 (+ 1232us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.170612 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.171931 11640 raft_consensus.cc:2379] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 4: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:38.172902 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.173906 11640 raft_consensus.cc:2379] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 5: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:38.175092 11640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41478 (request call id 930) took 1073 ms. Trace:
I20260809 15:52:38.121794 11015 raft_consensus.cc:2454] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:38.133798 13417 raft_consensus.cc:3037] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:38.161494 11009 rpcz_store.cc:276] 0809 15:52:36.896224 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:36.907236 (+ 11012us) service_pool.cc:227] Handling call
0809 15:52:36.907598 (+ 362us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:36.907983 (+ 385us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:36.908018 (+ 35us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:36.918004 (+ 9986us) log.cc:842] Serialized 26 byte log entry
0809 15:52:37.853784 (+935780us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:52:37.853837 (+ 53us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:37.854043 (+ 206us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:38.119541 (+265498us) raft_consensus.cc:1671] finished
0809 15:52:38.119558 (+ 17us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:38.119685 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":100,"raft.run_cpu_time_us":219,"raft.run_wall_time_us":57974,"spinlock_wait_cycles":7040,"thread_start_us":858312,"threads_started":1,"wal-append.queue_time_us":1118793}
I20260809 15:52:38.176507 11640 rpcz_store.cc:276] 0809 15:52:37.101301 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.172631 (+1071330us) service_pool.cc:227] Handling call
0809 15:52:38.175033 (+ 2402us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.179672 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:38.191395 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.190258 10891 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:38.193820 11635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 786) took 1271 ms. Trace:
I20260809 15:52:38.175484 10927 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
I20260809 15:52:38.201601 10811 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:38.146545 13553 raft_consensus.cc:686] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:38.211494 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.202800 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:38.205472 11015 raft_consensus.cc:1226] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:38.212878 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.215044 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:38.215687 10927 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:38.215978 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:38.194437 11635 rpcz_store.cc:276] 0809 15:52:36.922690 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.086270 (+163580us) service_pool.cc:227] Handling call
0809 15:52:37.109535 (+ 23265us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:37.118428 (+ 8893us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:37.118472 (+ 44us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:37.119028 (+ 556us) log.cc:842] Serialized 26 byte log entry
0809 15:52:37.811542 (+692514us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:38.193178 (+381636us) log.cc:842] Serialized 14 byte log entry
0809 15:52:38.193430 (+ 252us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.193530 (+ 100us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:38.193588 (+ 58us) raft_consensus.cc:1671] finished
0809 15:52:38.193606 (+ 18us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:38.193752 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8468,"raft.queue_time_us":9101,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":152,"thread_start_us":977909,"threads_started":2,"wal-append.queue_time_us":883146}
W20260809 15:52:38.221570 13644 log.cc:924] Time spent T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 1.694s user 0.004s sys 0.000s
I20260809 15:52:38.225224 10811 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:38.230988 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:38.232647 11009 raft_consensus.cc:1226] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260809 15:52:38.176076 13650 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:38.237411 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.201746 13597 raft_consensus.cc:2726] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:38.241729 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.175854 13611 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:38.200399 13689 raft_consensus.cc:2781] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:38.255823 11640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41488 (request call id 789) took 1135 ms. Trace:
I20260809 15:52:38.258677 11640 rpcz_store.cc:276] 0809 15:52:37.120608 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.240174 (+1119566us) service_pool.cc:227] Handling call
0809 15:52:38.255755 (+ 15581us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.207968 13690 raft_consensus.cc:2781] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:38.259646 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.230225 13260 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:38.159911 13326 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:38.250528 13553 consensus_queue.cc:237] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:38.264604 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:38.265626 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.236784 11634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41488 (request call id 788) took 1155 ms. Trace:
I20260809 15:52:38.268986 13571 raft_consensus.cc:2726] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:38.220614 13679 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:38.269531 10843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 651) took 2008 ms. Trace:
I20260809 15:52:38.245070 13646 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:38.274971 10929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 1963) took 1632 ms (client timeout 1736 ms). Trace:
W20260809 15:52:38.372262 10929 rpcz_store.cc:269] 0809 15:52:36.642892 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:36.987332 (+344440us) service_pool.cc:227] Handling call
0809 15:52:38.274774 (+1287442us) spinlock_profiling.cc:243] Waited 755 ms on lock 0x7b540015c2d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:38.274910 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1736578944}
I20260809 15:52:38.279531 13545 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:38.280761 11639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41478 (request call id 919) took 1188 ms. Trace:
I20260809 15:52:38.375837 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.376362 11639 rpcz_store.cc:276] 0809 15:52:37.092399 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.493871 (+401472us) service_pool.cc:227] Handling call
0809 15:52:38.280541 (+786670us) spinlock_profiling.cc:243] Waited 68.3 ms on lock 0x7b540066b554. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:38.280694 (+ 153us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":157097216}
I20260809 15:52:38.289796 11017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2207) took 1205 ms. Trace:
I20260809 15:52:38.257632 13689 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:38.300009 10927 raft_consensus.cc:2454] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:38.233448 13684 raft_consensus.cc:2781] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:52:38.385340 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.361541 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:38.363087 10843 rpcz_store.cc:276] 0809 15:52:36.260823 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:36.305008 (+ 44185us) service_pool.cc:227] Handling call
0809 15:52:36.305232 (+ 224us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:37.109962 (+804730us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.269219 (+1159257us) spinlock_profiling.cc:243] Waited 802 ms on lock 0x7b540022c754. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:38.269449 (+ 230us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1844304256}
I20260809 15:52:38.360054 11634 rpcz_store.cc:276] 0809 15:52:37.080968 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.125493 (+1044525us) service_pool.cc:227] Handling call
0809 15:52:38.236715 (+111222us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:52:38.369086 13662 consensus_meta.cc:287] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.568s user 0.003s sys 0.004s
I20260809 15:52:38.274480 11009 raft_consensus.cc:1226] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 4. Current term is 5. Ops: []
W20260809 15:52:38.429149 13693 log.cc:924] Time spent T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.166s user 0.001s sys 0.000s
I20260809 15:52:38.429205 13657 consensus_queue.cc:1046] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260809 15:52:38.431677 13657 raft_consensus.cc:3032] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:38.432297 13657 raft_consensus.cc:729] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:38.434633 10923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1888) took 2439 ms. Trace:
I20260809 15:52:38.434479 13662 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:38.435241 10923 rpcz_store.cc:276] 0809 15:52:35.995739 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:36.174301 (+178562us) service_pool.cc:227] Handling call
0809 15:52:36.187109 (+ 12808us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:36.240498 (+ 53389us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:36.240527 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:36.240944 (+ 417us) log.cc:842] Serialized 26 byte log entry
0809 15:52:37.476155 (+1235211us) spinlock_profiling.cc:243] Waited 4.94 ms on lock 0x7b1800250de0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f79665ad345 00007f796f3e4a39 00007f79665ca8bb 00007f79665c8cbf 00007f79665cae22 00007f79691745bb 00007f796917a063 00007f796917a411 00007f79694c75cf
0809 15:52:38.031531 (+555376us) raft_consensus.cc:1631] Marking committed up to 2
0809 15:52:38.031583 (+ 52us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.031670 (+ 87us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:38.434207 (+402537us) raft_consensus.cc:1671] finished
0809 15:52:38.434235 (+ 28us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:38.434562 (+ 327us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":53734,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":14156,"spinlock_wait_cycles":11367808,"thread_start_us":555116,"threads_started":1,"wal-append.queue_time_us":1989934}
I20260809 15:52:38.371083 13690 raft_consensus.cc:686] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:38.438488 13611 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:38.439020 13690 consensus_queue.cc:237] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:38.439050 11641 raft_consensus.cc:1226] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:38.442031 10923 raft_consensus.cc:1226] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260809 15:52:38.449092 10923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 699) took 2044 ms. Trace:
I20260809 15:52:38.378588 13545 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:38.233541 13691 raft_consensus.cc:2726] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 15:52:38.375718 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:38.475600 10929 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:38.377509 11017 rpcz_store.cc:276] 0809 15:52:37.084058 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.777471 (+693413us) service_pool.cc:227] Handling call
0809 15:52:37.792162 (+ 14691us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:38.166262 (+374100us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.289516 (+123254us) spinlock_profiling.cc:243] Waited 373 ms on lock 0x7b54001de054. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:38.289726 (+ 210us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":858667392}
I20260809 15:52:38.295439 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:38.240707 13503 consensus_queue.cc:1046] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.124s
I20260809 15:52:38.480715 11633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 780) took 1589 ms. Trace:
I20260809 15:52:38.380962 10811 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:38.481786 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 100925us
I20260809 15:52:38.363263 13423 consensus_queue.cc:237] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:38.388268 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:38.486516 10927 raft_consensus.cc:1799] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Handling vote request from an unknown peer d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:38.487073 10927 raft_consensus.cc:2354] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7.
I20260809 15:52:38.487488 10927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 1975) took 1660 ms. Trace:
I20260809 15:52:38.487807 10927 rpcz_store.cc:276] 0809 15:52:36.827402 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.388027 (+1560625us) service_pool.cc:227] Handling call
0809 15:52:38.487427 (+ 99400us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.358299 13260 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:38.489905 10929 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
I20260809 15:52:38.449699 10923 rpcz_store.cc:276] 0809 15:52:36.404360 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.437418 (+2033058us) service_pool.cc:227] Handling call
0809 15:52:38.437920 (+ 502us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:38.448786 (+ 10866us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.449035 (+ 249us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.298720 13504 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:38.462023 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.384079 13689 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
W20260809 15:52:38.427722 13424 consensus_meta.cc:287] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.561s user 0.009s sys 0.001s
I20260809 15:52:38.480820 10921 raft_consensus.cc:3037] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:38.382019 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:38.400825 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.426688 13684 raft_consensus.cc:686] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 7 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:38.492165 10811 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:38.497781 11639 raft_consensus.cc:3037] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:38.504233 11017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 516) took 1022 ms. Trace:
I20260809 15:52:38.504500 11635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41478 (request call id 946) took 1132 ms. Trace:
I20260809 15:52:38.505224 11635 rpcz_store.cc:276] 0809 15:52:37.372564 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.386429 (+1013865us) service_pool.cc:227] Handling call
0809 15:52:38.504427 (+117998us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.504765 11017 rpcz_store.cc:276] 0809 15:52:37.482191 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.485568 (+1003377us) service_pool.cc:227] Handling call
0809 15:52:38.503649 (+ 18081us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:38.503861 (+ 212us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:38.503892 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:38.503942 (+ 50us) raft_consensus.cc:1631] Marking committed up to 7
0809 15:52:38.503989 (+ 47us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.504051 (+ 62us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:38.504169 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:52:38.490936 10922 consensus_meta.cc:287] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.716s user 0.006s sys 0.007s
I20260809 15:52:38.509810 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.490294 13594 raft_consensus.cc:2781] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:38.521011 11640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41488 (request call id 790) took 1328 ms. Trace:
I20260809 15:52:38.525768 11640 rpcz_store.cc:276] 0809 15:52:37.192495 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.509523 (+1317028us) service_pool.cc:227] Handling call
0809 15:52:38.520944 (+ 11421us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.527041 11017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 517) took 1044 ms. Trace:
I20260809 15:52:38.527472 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:38.527619 11017 rpcz_store.cc:276] 0809 15:52:37.482645 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.507443 (+1024798us) service_pool.cc:227] Handling call
0809 15:52:38.526389 (+ 18946us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:38.526605 (+ 216us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:38.526639 (+ 34us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:38.526688 (+ 49us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:38.526732 (+ 44us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.526792 (+ 60us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:38.526953 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.529709 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.491479 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.508241 13691 raft_consensus.cc:2781] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:52:38.536882 11640 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:38.544569 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.542364 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:38.545471 11635 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:38.491043 13573 consensus_queue.cc:1046] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260809 15:52:38.548911 11017 raft_consensus.cc:1261] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 15:52:38.549831 11017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2001) took 1029 ms. Trace:
I20260809 15:52:38.550222 11017 rpcz_store.cc:276] 0809 15:52:37.520207 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.530012 (+1009805us) service_pool.cc:227] Handling call
0809 15:52:38.548482 (+ 18470us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:38.549535 (+ 1053us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.549770 (+ 235us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.441741 13657 consensus_queue.cc:260] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:38.567890 11640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41482 (request call id 614) took 1059 ms. Trace:
I20260809 15:52:38.521770 10922 raft_consensus.cc:2454] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:38.497150 13424 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:38.568964 11640 rpcz_store.cc:276] 0809 15:52:37.508839 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.544278 (+1035439us) service_pool.cc:227] Handling call
0809 15:52:38.567817 (+ 23539us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.574906 13260 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:38.575219 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.575999 11640 raft_consensus.cc:2379] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 9: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
W20260809 15:52:38.569236 11013 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.558s user 0.017s sys 0.048s
I20260809 15:52:38.539343 13594 raft_consensus.cc:686] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
W20260809 15:52:38.569356 10978 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.562s user 0.015s sys 0.054s
W20260809 15:52:38.581090 10978 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.574s user 0.015s sys 0.054s
W20260809 15:52:38.581477 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 574737us
I20260809 15:52:38.554471 13588 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:38.576635 11013 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.566s user 0.017s sys 0.048s
I20260809 15:52:38.582862 11013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 1994) took 1334 ms. Trace:
I20260809 15:52:38.584849 11013 rpcz_store.cc:276] 0809 15:52:37.248266 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.009999 (+761733us) service_pool.cc:227] Handling call
0809 15:52:38.010406 (+ 407us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:38.582520 (+572114us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:38.582563 (+ 43us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:38.582616 (+ 53us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:52:38.582652 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.582715 (+ 63us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:38.582813 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":165424,"raft.run_cpu_time_us":196,"raft.run_wall_time_us":34171,"thread_start_us":571646,"threads_started":1}
I20260809 15:52:38.589560 11639 raft_consensus.cc:2454] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:38.583493 11640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41482 (request call id 610) took 1129 ms. Trace:
I20260809 15:52:38.583334 10931 raft_consensus.cc:2454] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:38.590726 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.591044 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54224 (request call id 1914) took 1056 ms. Trace:
I20260809 15:52:38.591629 10931 rpcz_store.cc:276] 0809 15:52:37.534627 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.114450 (+579823us) service_pool.cc:227] Handling call
0809 15:52:38.590967 (+476517us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.592748 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:38.591133 10837 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.614s user 0.014s sys 0.021s
W20260809 15:52:38.593569 10837 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.617s user 0.014s sys 0.021s
I20260809 15:52:38.594020 11640 rpcz_store.cc:276] 0809 15:52:37.454492 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.574023 (+1119531us) service_pool.cc:227] Handling call
0809 15:52:38.583427 (+ 9404us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.595216 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.596408 11640 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:38.598534 11640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41488 (request call id 792) took 1331 ms. Trace:
I20260809 15:52:38.598815 10893 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:38.543015 11634 raft_consensus.cc:2454] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:38.584275 10978 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:38.598964 11640 rpcz_store.cc:276] 0809 15:52:37.267387 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.594928 (+1327541us) service_pool.cc:227] Handling call
0809 15:52:38.598486 (+ 3558us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:52:38.599928 10892 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.508s user 0.014s sys 0.014s
W20260809 15:52:38.599465 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 368594us
I20260809 15:52:38.370512 11016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 498) took 1936 ms. Trace:
I20260809 15:52:38.593513 10931 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:38.567124 13691 raft_consensus.cc:686] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 7 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:38.600142 10811 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:38.600718 11016 rpcz_store.cc:276] 0809 15:52:36.434492 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:36.724825 (+290333us) service_pool.cc:227] Handling call
0809 15:52:36.741827 (+ 17002us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:37.400204 (+658377us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.370238 (+970034us) spinlock_profiling.cc:243] Waited 658 ms on lock 0x7b5400000d54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:38.370447 (+ 209us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1512600320}
I20260809 15:52:38.602303 10893 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
W20260809 15:52:38.600245 10892 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.508s user 0.014s sys 0.014s
I20260809 15:52:38.605805 10811 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:38.608007 11639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41488 (request call id 791) took 1373 ms. Trace:
I20260809 15:52:38.608807 11639 rpcz_store.cc:276] 0809 15:52:37.234607 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.590431 (+1355824us) service_pool.cc:227] Handling call
0809 15:52:38.607954 (+ 17523us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.609385 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.544811 13640 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:38.617498 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:38.608263 10893 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:38.609925 13260 raft_consensus.cc:2781] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:38.607003 11634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41482 (request call id 615) took 1051 ms. Trace:
W20260809 15:52:38.611963 10926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 1957) took 2008 ms (client timeout 1817 ms). Trace:
W20260809 15:52:38.581291 10922 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 1982) took 1718 ms (client timeout 1581 ms). Trace:
W20260809 15:52:38.598889 13561 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.514s user 0.016s sys 0.012s
I20260809 15:52:38.606834 13646 raft_consensus.cc:2781] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:38.649384 11016 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50542
W20260809 15:52:38.661692 10926 rpcz_store.cc:269] 0809 15:52:36.603510 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.097775 (+494265us) service_pool.cc:227] Handling call
0809 15:52:38.611740 (+1513965us) spinlock_profiling.cc:243] Waited 72.5 ms on lock 0x7b540024dbd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:38.611886 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":166847360}
I20260809 15:52:38.661798 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:38.481215 11633 rpcz_store.cc:276] 0809 15:52:36.891249 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:36.894497 (+ 3248us) service_pool.cc:227] Handling call
0809 15:52:38.272497 (+1378000us) spinlock_profiling.cc:243] Waited 13.7 ms on lock 0x7b5800014198. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f7966593851 00007f7966593bce 000000000034aea9 00007f796e3bfa30 00007f796e4857b4 00007f796e485821 00007f796e4492b0 00007f796e4347f9 00007f796e4342a2 00007f7967b1a684
0809 15:52:38.273041 (+ 544us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:38.391014 (+117973us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:38.426188 (+ 35174us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:38.448494 (+ 22306us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:38.471877 (+ 23383us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.479150 (+ 7273us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:38.480636 (+ 1486us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":100246,"raft.queue_time_us":105221,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":7553,"spinlock_wait_cycles":31408384}
W20260809 15:52:38.662643 10978 connection.cc:232] RPC call timeout handler was delayed by -1.4655202169669792s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2066s for a response
I20260809 15:52:38.667290 13662 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:38.663172 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:38.666201 11016 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:38.690186 11016 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:38.668140 11641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 593) took 2876 ms. Trace:
I20260809 15:52:38.691004 11641 rpcz_store.cc:276] 0809 15:52:35.791380 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:35.791524 (+ 144us) service_pool.cc:227] Handling call
0809 15:52:37.518703 (+1727179us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:38.440895 (+922192us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.667508 (+226613us) spinlock_profiling.cc:243] Waited 920 ms on lock 0x7b54007dafd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:38.667959 (+ 451us) spinlock_profiling.cc:243] Waited 1.73 s on lock 0x7b54007dafd0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:38.668080 (+ 121us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":6087496832}
I20260809 15:52:38.692651 11641 raft_consensus.cc:1226] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:38.601329 13588 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:38.669219 10978 connection.cc:232] RPC call timeout handler was delayed by -1.4655202169669792s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2066s for a response
I20260809 15:52:38.669679 11013 raft_consensus.cc:2379] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 4: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:38.670940 13509 raft_consensus.cc:3032] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:38.671717 13640 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:38.666155 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:38.655761 13556 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:38.673151 10931 raft_consensus.cc:1226] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260809 15:52:38.713562 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1896) took 2255 ms. Trace:
I20260809 15:52:38.713965 10931 rpcz_store.cc:276] 0809 15:52:36.457925 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.648351 (+2190426us) service_pool.cc:227] Handling call
0809 15:52:38.672930 (+ 24579us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:38.713261 (+ 40331us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.713501 (+ 240us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.604117 13326 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:38.620322 13318 consensus_queue.cc:1046] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260809 15:52:38.678220 11635 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:38.679035 13691 consensus_queue.cc:237] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:38.682435 10926 raft_consensus.cc:1226] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 5. Current term is 6. Ops: []
W20260809 15:52:38.682476 11014 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.529s user 0.014s sys 0.074s
I20260809 15:52:38.613233 13571 raft_consensus.cc:2781] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:38.669906 13646 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:52:38.661115 13561 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.576s user 0.017s sys 0.012s
I20260809 15:52:38.708163 11639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41478 (request call id 936) took 1513 ms. Trace:
I20260809 15:52:38.664111 13504 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:38.716430 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:38.719386 11013 raft_consensus.cc:1226] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:38.657286 13661 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:38.664963 13260 raft_consensus.cc:686] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:38.663564 13619 consensus_queue.cc:1035] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.059s
I20260809 15:52:38.716276 13544 raft_consensus.cc:2726] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:38.706478 11641 raft_consensus.cc:1226] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:38.580819 13684 consensus_queue.cc:237] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:38.722962 10926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1898) took 2260 ms. Trace:
I20260809 15:52:38.872558 10926 rpcz_store.cc:276] 0809 15:52:36.462315 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.670237 (+2207922us) service_pool.cc:227] Handling call
0809 15:52:38.682224 (+ 11987us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:38.722692 (+ 40468us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.722898 (+ 206us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.703120 13509 raft_consensus.cc:729] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
W20260809 15:52:38.758587 13417 consensus_meta.cc:287] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.544s user 0.003s sys 0.008s
I20260809 15:52:38.876771 11641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 627) took 1004 ms. Trace:
I20260809 15:52:38.877833 11641 rpcz_store.cc:276] 0809 15:52:37.872119 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.705933 (+833814us) service_pool.cc:227] Handling call
0809 15:52:38.706267 (+ 334us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:38.876503 (+170236us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.876714 (+ 211us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.879240 11641 raft_consensus.cc:1226] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:38.827674 11639 rpcz_store.cc:276] 0809 15:52:37.194503 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.613139 (+1418636us) service_pool.cc:227] Handling call
0809 15:52:38.708098 (+ 94959us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.882041 11639 raft_consensus.cc:1226] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260809 15:52:38.734656 13318 raft_consensus.cc:3032] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:38.751562 13597 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:38.660851 13688 consensus_queue.cc:1046] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260809 15:52:38.832160 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 703) took 2366 ms. Trace:
I20260809 15:52:38.862674 11013 raft_consensus.cc:1226] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:38.648326 11634 rpcz_store.cc:276] 0809 15:52:37.555281 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.472694 (+917413us) service_pool.cc:227] Handling call
0809 15:52:38.603892 (+131198us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:52:38.743417 13465 consensus_meta.cc:287] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.648s user 0.009s sys 0.001s
I20260809 15:52:38.918687 10926 raft_consensus.cc:1261] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:38.919412 10926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1901) took 1553 ms. Trace:
I20260809 15:52:38.744041 13571 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 15:52:38.661015 10922 rpcz_store.cc:269] 0809 15:52:36.860695 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.465273 (+604578us) service_pool.cc:227] Handling call
0809 15:52:38.579005 (+1113732us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:38.875151 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:38.929358 11637 raft_consensus.cc:2362] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4. Re-sending same reply.
W20260809 15:52:38.929975 11637 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41478 (request call id 937) took 1705 ms (client timeout 1634 ms). Trace:
W20260809 15:52:38.930533 11637 rpcz_store.cc:269] 0809 15:52:37.225146 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.516227 (+291081us) service_pool.cc:227] Handling call
0809 15:52:38.874965 (+1358738us) spinlock_profiling.cc:243] Waited 2.49 ms on lock 0x7b5800014198. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f7966593851 00007f7966593702 00007f796f483b09 00007f796e3bf730 00007f796e48579b 00007f796e485821 00007f796e4351b0 00007f796e434a94 00007f7967b1cc04 00007f7967b1cb92
0809 15:52:38.929903 (+ 54938us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5719424}
W20260809 15:52:38.876233 10844 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 1966) took 1628 ms (client timeout 1636 ms). Trace:
W20260809 15:52:38.894208 11010 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.725s user 0.005s sys 0.043s
I20260809 15:52:38.902563 10931 rpcz_store.cc:276] 0809 15:52:36.465399 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.714846 (+2249447us) service_pool.cc:227] Handling call
0809 15:52:38.831504 (+116658us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:38.831723 (+ 219us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:38.831758 (+ 35us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:38.831837 (+ 79us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:38.831888 (+ 51us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.831953 (+ 65us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:38.832093 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:52:38.932796 11010 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.764s user 0.005s sys 0.043s
I20260809 15:52:38.829221 13646 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
W20260809 15:52:38.744210 11014 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.596s user 0.019s sys 0.075s
I20260809 15:52:38.919821 10926 rpcz_store.cc:276] 0809 15:52:37.365869 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.873668 (+1507799us) service_pool.cc:227] Handling call
0809 15:52:38.918317 (+ 44649us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:38.919128 (+ 811us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.919355 (+ 227us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:52:38.894567 13628 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.687s user 0.008s sys 0.024s
I20260809 15:52:38.703913 13572 raft_consensus.cc:2726] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:38.916246 13249 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:52:38.932698 10844 rpcz_store.cc:269] 0809 15:52:37.248360 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.253892 (+ 5532us) service_pool.cc:227] Handling call
0809 15:52:38.876004 (+1622112us) spinlock_profiling.cc:243] Waited 529 ms on lock 0x7b54001eee54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:38.876165 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1217253248}
I20260809 15:52:38.933606 11010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 1991) took 1807 ms. Trace:
I20260809 15:52:38.893101 13318 raft_consensus.cc:729] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:38.923430 13716 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:52:38.942360 13700 log.cc:924] Time spent T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.322s user 0.000s sys 0.001s
W20260809 15:52:38.942960 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 334631us
W20260809 15:52:38.941560 13628 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.764s user 0.008s sys 0.025s
I20260809 15:52:38.942456 13588 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:38.739082 13594 consensus_queue.cc:237] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:38.949182 11014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 1990) took 1829 ms. Trace:
I20260809 15:52:38.929726 13571 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:38.951368 13688 raft_consensus.cc:3032] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:38.953992 13548 raft_consensus.cc:2726] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:38.872598 13661 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:38.808683 10811 leader_election.cc:334] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.709s (SENT)
I20260809 15:52:38.980865 10811 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:38.982020 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 173414us
W20260809 15:52:38.937752 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 329532us
I20260809 15:52:38.984412 10893 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:38.964238 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 270561us
I20260809 15:52:38.923388 13465 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:38.950673 11009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2213) took 1471 ms. Trace:
I20260809 15:52:38.988596 11009 rpcz_store.cc:276] 0809 15:52:37.479404 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.426068 (+946664us) service_pool.cc:227] Handling call
0809 15:52:38.426493 (+ 425us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:38.426734 (+ 241us) raft_consensus.cc:1520] Early marking committed up to index 2
0809 15:52:38.475075 (+ 48341us) log.cc:842] Serialized 14 byte log entry
0809 15:52:38.950293 (+475218us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:38.950354 (+ 61us) raft_consensus.cc:1631] Marking committed up to 2
0809 15:52:38.950418 (+ 64us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.950489 (+ 71us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:38.950611 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4904,"prepare.queue_time_us":48634,"prepare.run_cpu_time_us":85,"prepare.run_wall_time_us":86,"raft.queue_time_us":9035,"spinlock_wait_cycles":9360640,"thread_start_us":469684,"threads_started":1,"wal-append.queue_time_us":495546}
I20260809 15:52:38.990620 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:38.992043 13723 raft_consensus.cc:1050] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3: attempting to promote NON_VOTER c7aa762318cb4e97875333f10679a9f9 to VOTER
I20260809 15:52:38.917701 13597 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:38.993651 13704 log.cc:924] Time spent T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.077s user 0.000s sys 0.000s
I20260809 15:52:38.994277 13597 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:38.951737 13424 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:38.997403 13661 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:38.997831 13465 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:38.998621 11015 raft_consensus.cc:1226] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:38.998962 10847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2132) took 1774 ms. Trace:
I20260809 15:52:38.999321 10847 rpcz_store.cc:276] 0809 15:52:37.226644 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.242666 (+ 16022us) service_pool.cc:227] Handling call
0809 15:52:37.243192 (+ 526us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:37.873340 (+630148us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:37.873386 (+ 46us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:37.873960 (+ 574us) log.cc:842] Serialized 26 byte log entry
0809 15:52:38.485790 (+611830us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:38.945215 (+459425us) log.cc:842] Serialized 14 byte log entry
0809 15:52:38.953744 (+ 8529us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.955466 (+ 1722us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:38.998757 (+ 43291us) raft_consensus.cc:1671] finished
0809 15:52:38.998774 (+ 17us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:38.998905 (+ 131us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":7750,"prepare.queue_time_us":229945,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"raft.queue_time_us":31652,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":6286,"thread_start_us":1549511,"threads_started":3,"wal-append.queue_time_us":815534}
I20260809 15:52:39.000944 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:39.002039 11009 raft_consensus.cc:2396] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260809 15:52:39.002426 10893 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:39.003500 13424 raft_consensus.cc:2726] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:38.943727 11010 rpcz_store.cc:276] 0809 15:52:37.125788 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.832887 (+707099us) service_pool.cc:227] Handling call
0809 15:52:37.833160 (+ 273us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:37.833346 (+ 186us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 15:52:38.159660 (+326314us) log.cc:842] Serialized 14 byte log entry
0809 15:52:38.933282 (+773622us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:38.933334 (+ 52us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:38.933378 (+ 44us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.933437 (+ 59us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:38.933547 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":420182,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":55,"raft.queue_time_us":168,"thread_start_us":1076442,"threads_started":2}
I20260809 15:52:38.978044 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:39.005308 11642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41488 (request call id 787) took 2019 ms (client timeout 2153 ms). Trace:
W20260809 15:52:38.978338 11011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50528 (request call id 1976) took 2612 ms (client timeout 2647 ms). Trace:
I20260809 15:52:38.900184 13417 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:38.719507 13556 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:38.787185 11016 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:39.012691 11012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 1992) took 1810 ms. Trace:
I20260809 15:52:39.013150 11012 rpcz_store.cc:276] 0809 15:52:37.202745 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.839987 (+637242us) service_pool.cc:227] Handling call
0809 15:52:37.843200 (+ 3213us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:37.843651 (+ 451us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:37.843678 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:37.851776 (+ 8098us) log.cc:842] Serialized 26 byte log entry
0809 15:52:38.266510 (+414734us) raft_consensus.cc:1631] Marking committed up to 2
0809 15:52:38.266572 (+ 62us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.266652 (+ 80us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:39.012453 (+745801us) raft_consensus.cc:1671] finished
0809 15:52:39.012481 (+ 28us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:39.012623 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":2642,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":16106,"thread_start_us":392893,"threads_started":1,"wal-append.queue_time_us":713985}
I20260809 15:52:38.972062 13688 raft_consensus.cc:729] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
W20260809 15:52:39.005987 11011 rpcz_store.cc:269] 0809 15:52:36.366193 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:36.507646 (+141453us) service_pool.cc:227] Handling call
0809 15:52:38.978187 (+2470541us) spinlock_profiling.cc:243] Waited 67.7 ms on lock 0x7b54003f8754. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:38.978285 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":155706240}
W20260809 15:52:39.005694 11642 rpcz_store.cc:269] 0809 15:52:36.985520 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.960947 (+1975427us) service_pool.cc:227] Handling call
0809 15:52:39.005248 (+ 44301us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:39.018432 10923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1892) took 2561 ms. Trace:
I20260809 15:52:39.016765 11016 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:38.952154 13249 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:38.987780 10978 connection.cc:232] RPC call timeout handler was delayed by -1.4313202169723809s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21040000000000003s for a response
W20260809 15:52:39.028913 10978 connection.cc:232] RPC call timeout handler was delayed by -1.4187202169559896s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2118s for a response
W20260809 15:52:39.029342 10978 connection.cc:232] RPC call timeout handler was delayed by -1.3827202169923112s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2158s for a response
I20260809 15:52:38.986603 13504 raft_consensus.cc:2726] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:39.031647 10931 raft_consensus.cc:1226] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:39.032208 11016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:50542 (request call id 2216) took 1246 ms. Trace:
I20260809 15:52:38.947244 13509 consensus_queue.cc:260] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:39.032699 11016 rpcz_store.cc:276] 0809 15:52:37.781023 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.605812 (+824789us) service_pool.cc:227] Handling call
0809 15:52:39.020695 (+414883us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:39.019253 10923 rpcz_store.cc:276] 0809 15:52:36.456801 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.490728 (+2033927us) service_pool.cc:227] Handling call
0809 15:52:38.508589 (+ 17861us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:38.523128 (+ 14539us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:38.523170 (+ 42us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:38.523682 (+ 512us) log.cc:842] Serialized 26 byte log entry
0809 15:52:38.953448 (+429766us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:52:38.953496 (+ 48us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.953587 (+ 91us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:39.018188 (+ 64601us) raft_consensus.cc:1671] finished
0809 15:52:39.018214 (+ 26us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:39.018365 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2005,"raft.queue_time_us":14454,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":12795,"thread_start_us":397447,"threads_started":1,"wal-append.queue_time_us":447823}
W20260809 15:52:39.037245 10978 connection.cc:232] RPC call timeout handler was delayed by -1.0530800429987721s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1524s for a response
I20260809 15:52:39.044626 11011 raft_consensus.cc:1226] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:39.046428 13243 consensus_queue.cc:1046] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:39.007956 13708 consensus_queue.cc:1046] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260809 15:52:39.049676 13417 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:39.051400 13545 consensus_meta.cc:287] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.570s user 0.008s sys 0.001s
I20260809 15:52:39.051888 13545 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:39.005831 13424 consensus_queue.cc:1035] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.018s
W20260809 15:52:39.056842 13701 log.cc:924] Time spent T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.109s user 0.000s sys 0.000s
I20260809 15:52:38.973261 13260 consensus_queue.cc:237] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:39.059000 10837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2129) took 1859 ms. Trace:
I20260809 15:52:39.059437 10837 rpcz_store.cc:276] 0809 15:52:37.199857 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.233914 (+ 34057us) service_pool.cc:227] Handling call
0809 15:52:37.245793 (+ 11879us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:37.877964 (+632171us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:37.878007 (+ 43us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:37.893347 (+ 15340us) log.cc:842] Serialized 26 byte log entry
0809 15:52:38.594288 (+700941us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:52:38.965178 (+370890us) log.cc:842] Serialized 14 byte log entry
0809 15:52:38.974972 (+ 9794us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.975072 (+ 100us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:39.058744 (+ 83672us) raft_consensus.cc:1671] finished
0809 15:52:39.058770 (+ 26us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:39.058920 (+ 150us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":11277,"prepare.queue_time_us":146763,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":30390,"raft.run_cpu_time_us":212,"raft.run_wall_time_us":7515,"thread_start_us":1596285,"threads_started":3,"wal-append.queue_time_us":903615}
I20260809 15:52:39.062268 11633 raft_consensus.cc:1226] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
W20260809 15:52:39.063697 10921 consensus_meta.cc:287] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.543s user 0.008s sys 0.002s
I20260809 15:52:39.064224 10921 raft_consensus.cc:2454] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:39.065322 13249 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:39.066210 13711 log.cc:924] Time spent T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.142s user 0.001s sys 0.000s
W20260809 15:52:39.067750 13705 log.cc:924] Time spent T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.194s user 0.000s sys 0.001s
I20260809 15:52:39.030629 13679 raft_consensus.cc:482] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:39.082693 11010 raft_consensus.cc:1226] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 2. Current term is 3. Ops: []
W20260809 15:52:39.084519 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 44932.5us
I20260809 15:52:39.086491 11641 raft_consensus.cc:1226] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:38.965747 11014 rpcz_store.cc:276] 0809 15:52:37.120053 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.817362 (+697309us) service_pool.cc:227] Handling call
0809 15:52:37.837165 (+ 19803us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:37.837342 (+ 177us) raft_consensus.cc:1520] Early marking committed up to index 3
0809 15:52:38.147667 (+310325us) log.cc:842] Serialized 14 byte log entry
0809 15:52:38.948874 (+801207us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:38.948920 (+ 46us) raft_consensus.cc:1631] Marking committed up to 3
0809 15:52:38.948967 (+ 47us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.949029 (+ 62us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:38.949130 (+ 101us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":22551,"prepare.queue_time_us":429012,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"raft.queue_time_us":16907,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":8304,"thread_start_us":1079100,"threads_started":2,"wal-append.queue_time_us":781369}
I20260809 15:52:39.092410 11012 raft_consensus.cc:1226] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:39.069551 13545 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:39.070791 13688 consensus_queue.cc:260] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:39.075215 13710 log.cc:924] Time spent T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.158s user 0.003s sys 0.000s
I20260809 15:52:39.075585 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:39.075784 10927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 702) took 2620 ms. Trace:
I20260809 15:52:39.075379 10923 raft_consensus.cc:1226] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:38.957737 13716 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:39.035301 11632 consensus_meta.cc:287] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.821s user 0.011s sys 0.002s
W20260809 15:52:39.079927 10839 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 1968) took 1800 ms (client timeout 2092 ms). Trace:
I20260809 15:52:39.131827 10927 rpcz_store.cc:276] 0809 15:52:36.455693 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.488535 (+2032842us) service_pool.cc:227] Handling call
0809 15:52:38.488920 (+ 385us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:38.496339 (+ 7419us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:38.496369 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:38.508619 (+ 12250us) log.cc:842] Serialized 26 byte log entry
0809 15:52:38.916437 (+407818us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:38.916490 (+ 53us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:38.916581 (+ 91us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:39.075565 (+158984us) raft_consensus.cc:1671] finished
0809 15:52:39.075587 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:39.075728 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":13700,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":18889,"thread_start_us":371826,"threads_started":1,"wal-append.queue_time_us":456571}
I20260809 15:52:39.132756 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:39.136900 10839 rpcz_store.cc:269] 0809 15:52:37.279748 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.309015 (+ 29267us) service_pool.cc:227] Handling call
0809 15:52:39.079744 (+1770729us) spinlock_profiling.cc:243] Waited 244 ms on lock 0x7b540005a0d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:39.079869 (+ 125us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":561766656}
I20260809 15:52:39.137415 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:39.137763 11641 raft_consensus.cc:2379] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 4: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:39.137257 10923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1911) took 1635 ms. Trace:
W20260809 15:52:39.095758 13689 consensus_meta.cc:287] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.587s user 0.008s sys 0.000s
I20260809 15:52:39.085299 10978 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
W20260809 15:52:39.132126 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 469370us
I20260809 15:52:39.114990 11010 raft_consensus.cc:1226] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:39.089732 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:39.096597 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:39.136544 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:39.143766 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:39.144243 11639 raft_consensus.cc:2379] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 2: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:39.096346 10929 raft_consensus.cc:1226] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:39.138558 10923 rpcz_store.cc:276] 0809 15:52:37.502061 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:39.040172 (+1538111us) service_pool.cc:227] Handling call
0809 15:52:39.075204 (+ 35032us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:39.136748 (+ 61544us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:39.137178 (+ 430us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:39.144889 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:39.152895 10927 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:39.157913 11011 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:39.131634 13249 raft_consensus.cc:3032] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:39.092818 13708 raft_consensus.cc:3032] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:39.110755 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:39.164496 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 79292.4us
I20260809 15:52:39.186508 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:39.187683 10978 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:39.082218 10921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54224 (request call id 1916) took 1463 ms. Trace:
I20260809 15:52:39.160941 10893 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:39.188968 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:39.188812 10978 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:39.192505 10893 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:39.159929 13689 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:39.185431 13249 raft_consensus.cc:729] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:39.202145 11018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2206) took 2118 ms. Trace:
I20260809 15:52:39.202319 11017 raft_consensus.cc:1226] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:39.202553 11018 rpcz_store.cc:276] 0809 15:52:37.083757 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.769457 (+685700us) service_pool.cc:227] Handling call
0809 15:52:37.856347 (+ 86890us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:37.864737 (+ 8390us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:39.201950 (+1337213us) spinlock_profiling.cc:243] Waited 86.6 ms on lock 0x7b540004d3d0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:39.202071 (+ 121us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":199219456}
I20260809 15:52:39.201731 13249 consensus_queue.cc:260] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:39.204290 13249 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:39.204924 11017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2220) took 1108 ms. Trace:
I20260809 15:52:39.205348 11017 rpcz_store.cc:276] 0809 15:52:38.096310 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.900437 (+804127us) service_pool.cc:227] Handling call
0809 15:52:38.931772 (+ 31335us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:39.202868 (+271096us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:39.204669 (+ 1801us) spinlock_profiling.cc:243] Waited 270 ms on lock 0x7b5400180454. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:39.204860 (+ 191us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":621366528}
I20260809 15:52:39.207620 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:39.105787 13544 consensus_queue.cc:1046] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.084s
I20260809 15:52:39.142303 11632 raft_consensus.cc:2454] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:39.188789 13708 raft_consensus.cc:729] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:39.212975 11018 raft_consensus.cc:1261] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 4 index: 3. (index mismatch)
I20260809 15:52:39.215184 10927 raft_consensus.cc:2354] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5.
I20260809 15:52:39.137856 13299 consensus_queue.cc:1046] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.064s
I20260809 15:52:39.189980 13465 raft_consensus.cc:2781] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:39.207018 13426 raft_consensus.cc:2726] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:39.191056 13703 raft_consensus.cc:2726] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:52:39.224727 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:39.236464 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:39.219332 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 235923us
I20260809 15:52:39.228253 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:39.110483 13679 raft_consensus.cc:504] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:39.198606 10978 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:39.185199 13334 consensus_queue.cc:1046] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.059s
I20260809 15:52:39.209426 13702 raft_consensus.cc:2781] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:39.245538 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:39.219451 13689 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:39.280233 13722 raft_consensus.cc:2726] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:39.357067 10848 raft_consensus.cc:1226] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
W20260809 15:52:39.281761 13727 log.cc:924] Time spent T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.070s user 0.001s sys 0.000s
I20260809 15:52:39.281968 13513 raft_consensus.cc:3037] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:39.286836 13653 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:39.292112 13556 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:39.294929 10841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:45572 (request call id 926) took 1600 ms. Trace:
I20260809 15:52:39.295018 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:39.363175 11017 raft_consensus.cc:2396] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260809 15:52:39.362720 10841 rpcz_store.cc:276] 0809 15:52:37.694574 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.694702 (+ 128us) service_pool.cc:227] Handling call
0809 15:52:39.294194 (+1599492us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b54001c61d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478af0 00007f79694fbf2a 00007f796e438c6e 00007f7967b2f807 00007f7967b2f792 00007f7967b2f6c1 00007f7967b2f63c 00007f7967b2e8f2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:39.294815 (+ 621us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2896658688}
I20260809 15:52:39.295744 13522 raft_consensus.cc:3032] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:39.364478 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:39.364995 11640 raft_consensus.cc:1203] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Deduplicated request from leader. Original: 4.7->[5.8-5.8] Dedup: 5.8->[]
I20260809 15:52:39.365618 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:39.299890 13723 consensus_queue.cc:237] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 4, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:39.301105 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:39.227064 13465 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:39.302138 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:39.304325 10926 raft_consensus.cc:1226] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:39.304906 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:39.247346 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 54947.8us
I20260809 15:52:39.305572 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:39.376302 11639 raft_consensus.cc:2379] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 4: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:39.376626 10893 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:39.376879 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:39.307686 10920 raft_consensus.cc:1226] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:39.380118 10926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1904) took 2012 ms. Trace:
I20260809 15:52:39.343514 13702 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:52:39.310621 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 97332.7us
I20260809 15:52:39.383502 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:39.312412 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:39.388613 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:39.388768 10839 raft_consensus.cc:2454] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:39.323062 10844 raft_consensus.cc:1226] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:39.391767 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:39.392455 10844 raft_consensus.cc:3037] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:39.329960 10843 raft_consensus.cc:1226] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:39.333866 10840 raft_consensus.cc:1226] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260809 15:52:39.397624 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:39.399173 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:39.400697 10927 raft_consensus.cc:2379] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 3: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:39.401000 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:39.356117 13703 raft_consensus.cc:2781] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:39.405067 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:39.406399 10927 raft_consensus.cc:3037] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:39.405069 10840 raft_consensus.cc:1226] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:39.345158 13571 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:39.338351 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 179832us
I20260809 15:52:39.417359 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:39.338413 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 99809.7us
I20260809 15:52:39.339510 10847 raft_consensus.cc:1226] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:39.399021 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:39.420367 11639 raft_consensus.cc:2379] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 4: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:39.344549 10838 raft_consensus.cc:1226] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 7 LEADER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260809 15:52:39.421183 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:39.386173 13653 raft_consensus.cc:2726] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:39.191179 10921 rpcz_store.cc:276] 0809 15:52:37.618717 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.214648 (+595931us) service_pool.cc:227] Handling call
0809 15:52:39.080763 (+866115us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":32128}
W20260809 15:52:39.367913 13728 log.cc:924] Time spent T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.120s user 0.000s sys 0.001s
I20260809 15:52:39.371026 10923 raft_consensus.cc:2354] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3.
I20260809 15:52:39.378830 10893 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:39.380489 10926 rpcz_store.cc:276] 0809 15:52:37.368097 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.936185 (+1568088us) service_pool.cc:227] Handling call
0809 15:52:39.303967 (+367782us) spinlock_profiling.cc:243] Waited 920 us on lock 0x7b58000d1498. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f7966593851 00007f7966593702 00007f796f483b09 00007f796e3bf730 00007f796e48579b 00007f796e485821 00007f796e4492b0 00007f796e4347f9 00007f796e4342a2 00007f7967b1a684
0809 15:52:39.304161 (+ 194us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:39.379861 (+ 75700us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:39.380065 (+ 204us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2116992}
I20260809 15:52:39.220784 13690 consensus_queue.cc:1046] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.089s
I20260809 15:52:39.380920 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:39.381825 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:39.413304 11606 connection.cc:232] RPC call timeout handler was delayed by -1.3862300559994765s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18520000000000003s for a response
I20260809 15:52:39.368968 13522 raft_consensus.cc:729] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:39.438009 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:39.355717 13419 consensus_queue.cc:1046] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 2.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.041s
I20260809 15:52:39.382468 13465 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:39.397672 13702 raft_consensus.cc:3037] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:39.365724 11640 raft_consensus.cc:1226] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 5. Current term is 6. Ops: [5.8-5.8]
I20260809 15:52:39.382692 13504 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:39.425504 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:39.426862 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:39.389513 13646 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:39.435225 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:39.436539 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:39.457895 10840 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:52:39.412925 10920 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 691) took 3077 ms. Trace:
I20260809 15:52:39.441605 13575 consensus_queue.cc:1046] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.018s
I20260809 15:52:39.459175 10920 rpcz_store.cc:276] 0809 15:52:36.335288 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:36.344378 (+ 9090us) service_pool.cc:227] Handling call
0809 15:52:38.288237 (+1943859us) spinlock_profiling.cc:243] Waited 7.01 ms on lock 0x7b58000d1498. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f7966593851 00007f7966593702 00007f796f483b09 00007f796e3bf730 00007f796e48579b 00007f796e485821 00007f796e4492b0 00007f796e4347f9 00007f796e4342a2 00007f7967b1a684
0809 15:52:39.307132 (+1018895us) spinlock_profiling.cc:243] Waited 6.87 ms on lock 0x7b58000d1498. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f7966593b7c 00007f796659373b 00007f796f483b09 00007f796e3bf730 00007f796e48579b 00007f796e485821 00007f796e4492b0 00007f796e4347f9 00007f796e4342a2 00007f7967b1a684
0809 15:52:39.307410 (+ 278us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:39.396347 (+ 88937us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:39.412851 (+ 16504us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":31931520}
I20260809 15:52:39.461145 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:39.410876 13722 consensus_queue.cc:1035] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.051s
I20260809 15:52:39.462193 10841 raft_consensus.cc:2379] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 3: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:39.458179 13243 consensus_queue.cc:1046] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.005s
I20260809 15:52:39.464274 13243 raft_consensus.cc:3032] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:39.463811 13522 consensus_queue.cc:260] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:39.464655 13243 raft_consensus.cc:729] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:39.465570 10734 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33062 (request call id 393) took 10476 ms. Trace:
I20260809 15:52:39.466136 13504 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:39.467509 13344 raft_consensus.cc:2726] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 15:52:39.458374 13661 consensus_queue.cc:1046] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.010s
I20260809 15:52:39.435274 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:39.475948 13731 consensus_queue.cc:1035] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.023s
I20260809 15:52:39.478981 13464 raft_consensus.cc:3032] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:39.479367 13464 raft_consensus.cc:729] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:39.474889 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:39.480161 13464 consensus_queue.cc:260] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:39.339090 13708 consensus_queue.cc:260] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:39.481654 11018 raft_consensus.cc:1226] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: [3.3-3.3]
I20260809 15:52:39.440380 10838 raft_consensus.cc:1226] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:39.439167 10811 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:39.434023 13167 raft_consensus.cc:3037] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:39.441748 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:39.483027 13679 leader_election.cc:288] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:39.445986 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 67252.2us
I20260809 15:52:39.483898 10893 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:39.337401 11011 raft_consensus.cc:2454] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:39.484848 13722 raft_consensus.cc:2781] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:39.485229 13722 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:39.485574 13722 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:39.486601 13716 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:39.490331 11014 raft_consensus.cc:1226] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 4. Current term is 5. Ops: [4.3-4.3]
I20260809 15:52:39.493221 10926 raft_consensus.cc:1226] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:39.470845 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:39.493804 10926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1912) took 1960 ms. Trace:
I20260809 15:52:39.472160 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:39.494156 10926 rpcz_store.cc:276] 0809 15:52:37.533465 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:39.437984 (+1904519us) service_pool.cc:227] Handling call
0809 15:52:39.493067 (+ 55083us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:39.493593 (+ 526us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:39.493756 (+ 163us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:39.494817 10843 raft_consensus.cc:2396] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260809 15:52:39.495729 10929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1893) took 3038 ms. Trace:
I20260809 15:52:39.496109 10929 rpcz_store.cc:276] 0809 15:52:36.457154 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.491280 (+2034126us) service_pool.cc:227] Handling call
0809 15:52:38.491619 (+ 339us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:39.156129 (+664510us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:39.495015 (+338886us) spinlock_profiling.cc:243] Waited 5.81 ms on lock 0x7b5400449920. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f7969483730 00007f79694f6b42 00007f79694f2f26 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772
0809 15:52:39.495500 (+ 485us) spinlock_profiling.cc:243] Waited 604 ms on lock 0x7b540015c2d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:39.495668 (+ 168us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1403634944}
I20260809 15:52:39.497551 11013 raft_consensus.cc:1226] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:39.498493 13646 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:39.498695 11013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2222) took 1366 ms. Trace:
I20260809 15:52:39.499050 11013 rpcz_store.cc:276] 0809 15:52:38.131993 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.939749 (+807756us) service_pool.cc:227] Handling call
0809 15:52:38.960890 (+ 21141us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:39.497966 (+537076us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:39.498497 (+ 531us) spinlock_profiling.cc:243] Waited 536 ms on lock 0x7b54004204d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:39.498647 (+ 150us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1233811328}
I20260809 15:52:39.502813 10891 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:39.504436 13423 raft_consensus.cc:2781] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:39.480700 13571 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:39.505052 13423 raft_consensus.cc:686] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:39.506150 13423 consensus_queue.cc:237] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:39.502962 11640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41478 (request call id 921) took 2404 ms. Trace:
I20260809 15:52:39.508523 11640 rpcz_store.cc:276] 0809 15:52:37.098646 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.613931 (+1515285us) service_pool.cc:227] Handling call
0809 15:52:38.614248 (+ 317us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:39.456532 (+842284us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:39.502686 (+ 46154us) spinlock_profiling.cc:243] Waited 751 ms on lock 0x7b54005203d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:39.502890 (+ 204us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1726208768}
I20260809 15:52:39.513016 13540 raft_consensus.cc:3032] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:39.513576 13540 raft_consensus.cc:729] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:39.512831 13318 consensus_queue.cc:260] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:39.514503 13540 consensus_queue.cc:260] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:39.515777 13540 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:39.521307 13575 raft_consensus.cc:3032] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:39.521803 13575 raft_consensus.cc:729] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
W20260809 15:52:39.522532 10924 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54224 (request call id 1905) took 2136 ms (client timeout 1802 ms). Trace:
I20260809 15:52:39.523288 13661 raft_consensus.cc:3032] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:39.523787 13661 raft_consensus.cc:729] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:39.524572 13688 raft_consensus.cc:3037] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:39.524734 13661 consensus_queue.cc:260] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:39.523052 10924 rpcz_store.cc:269] 0809 15:52:37.387391 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:39.522326 (+2134935us) service_pool.cc:214] Skipping call since client already timed out
0809 15:52:39.522450 (+ 124us) inbound_call.cc:173] Queueing failure response
Metrics: {}
I20260809 15:52:39.527498 13509 raft_consensus.cc:3037] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:39.527645 10924 raft_consensus.cc:1226] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260809 15:52:39.529250 10924 raft_consensus.cc:1226] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:39.531216 10842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 1962) took 2303 ms. Trace:
I20260809 15:52:39.531486 10837 raft_consensus.cc:1226] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 7 LEADER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260809 15:52:39.531692 10842 rpcz_store.cc:276] 0809 15:52:37.228310 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.233201 (+ 4891us) service_pool.cc:227] Handling call
0809 15:52:39.530834 (+2297633us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b54000596d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:39.531136 (+ 302us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":514197376}
I20260809 15:52:39.411103 13703 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:39.535727 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:39.536410 10923 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:39.561673 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:39.561905 13318 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:39.465523 13243 consensus_queue.cc:260] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:39.560755 13703 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:39.561406 13661 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
W20260809 15:52:39.588138 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 143892us
I20260809 15:52:39.589758 11606 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
W20260809 15:52:39.616536 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 132717us
I20260809 15:52:39.617296 10893 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:39.495131 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:39.618642 10893 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
W20260809 15:52:39.618824 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 179723us
I20260809 15:52:39.466012 10734 rpcz_store.cc:276] 0809 15:52:28.989299 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:28.989441 (+ 142us) service_pool.cc:227] Handling call
0809 15:52:39.465473 (+10476032us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 15:52:32.080202 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 15:52:32.080748 (+ 546us) write_op.cc:432] Acquiring schema lock in shared mode
0809 15:52:32.080787 (+ 39us) write_op.cc:435] Acquired schema lock
0809 15:52:32.080803 (+ 16us) tablet.cc:660] Decoding operations
0809 15:52:32.081128 (+ 325us) write_op.cc:620] Acquiring the partition lock for write op
0809 15:52:32.081209 (+ 81us) write_op.cc:641] Partition lock acquired for write op
0809 15:52:32.081237 (+ 28us) tablet.cc:683] Acquiring locks for 1 operations
0809 15:52:32.081422 (+ 185us) tablet.cc:699] Row locks acquired
0809 15:52:32.081440 (+ 18us) write_op.cc:260] PREPARE: finished
0809 15:52:32.081596 (+ 156us) write_op.cc:270] Start()
0809 15:52:32.081703 (+ 107us) write_op.cc:276] Timestamp: P: 1786290752081548 usec, L: 0
0809 15:52:32.081725 (+ 22us) op_driver.cc:331] REPLICATION: starting
0809 15:52:32.140412 (+ 58687us) log.cc:842] Serialized 527 byte log entry
0809 15:52:34.347367 (+2206955us) spinlock_profiling.cc:243] Waited 21 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:52:35.743935 (+1396568us) op_driver.cc:444] REPLICATION: finished
0809 15:52:36.264206 (+520271us) write_op.cc:301] APPLY: starting
0809 15:52:36.264296 (+ 90us) tablet.cc:1365] starting BulkCheckPresence
0809 15:52:36.264398 (+ 102us) tablet.cc:1368] finished BulkCheckPresence
0809 15:52:36.264421 (+ 23us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 15:52:36.264703 (+ 282us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 15:52:36.264908 (+ 205us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=1
0809 15:52:36.264969 (+ 61us) write_op.cc:312] APPLY: finished
0809 15:52:36.265190 (+ 221us) log.cc:842] Serialized 23 byte log entry
0809 15:52:36.265352 (+ 162us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:52:36.265470 (+ 118us) write_op.cc:454] Released schema lock
0809 15:52:36.265529 (+ 59us) write_op.cc:341] FINISH: Updating metrics
0809 15:52:36.265879 (+ 350us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:52:36.265909 (+ 30us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":965551,"reported_tablets":4,"child_traces":[["op",{"apply.queue_time_us":502756,"apply.run_cpu_time_us":1989,"apply.run_wall_time_us":2685,"num_ops":1,"prepare.queue_time_us":183424,"prepare.run_cpu_time_us":9089,"prepare.run_wall_time_us":534180,"raft.queue_time_us":1318223,"raft.run_cpu_time_us":12225,"raft.run_wall_time_us":431551,"replication_time_us":3632475,"spinlock_wait_cycles":48362240,"thread_start_us":2351893,"threads_started":4,"wal-append.queue_time_us":511506,"wal-append.run_cpu_time_us":67789,"wal-append.run_wall_time_us":4563190}]]}
W20260809 15:52:39.595285 13607 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.311s user 0.012s sys 0.028s
I20260809 15:52:39.619688 11009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 522) took 1383 ms. Trace:
I20260809 15:52:39.638581 11009 rpcz_store.cc:276] 0809 15:52:38.236505 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:39.003578 (+767073us) service_pool.cc:227] Handling call
0809 15:52:39.003835 (+ 257us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:39.004090 (+ 255us) raft_consensus.cc:1520] Early marking committed up to index 8
0809 15:52:39.208608 (+204518us) log.cc:842] Serialized 14 byte log entry
0809 15:52:39.619379 (+410771us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:39.619429 (+ 50us) raft_consensus.cc:1631] Marking committed up to 8
0809 15:52:39.619475 (+ 46us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:39.619536 (+ 61us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:39.619635 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":431639,"prepare.run_cpu_time_us":30,"prepare.run_wall_time_us":31,"raft.queue_time_us":92,"thread_start_us":614637,"threads_started":2}
I20260809 15:52:39.635463 10844 raft_consensus.cc:2454] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:39.640265 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:39.641199 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:39.641933 10842 raft_consensus.cc:2354] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5.
I20260809 15:52:39.642642 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:39.644680 10893 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:39.646184 10893 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:39.651507 10926 raft_consensus.cc:1226] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:39.652021 11009 raft_consensus.cc:1226] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:39.659659 10893 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
W20260809 15:52:39.638512 13607 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.354s user 0.012s sys 0.029s
I20260809 15:52:39.635334 13731 raft_consensus.cc:2781] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:39.672308 13731 raft_consensus.cc:686] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:39.673168 13731 consensus_queue.cc:237] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:39.653409 13575 consensus_queue.cc:260] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:39.664894 13503 consensus_queue.cc:1046] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.010s
I20260809 15:52:39.635578 13419 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:39.675168 13575 raft_consensus.cc:3037] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:39.675515 13419 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:39.667063 10840 raft_consensus.cc:2396] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260809 15:52:39.638226 13656 raft_consensus.cc:3037] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:39.677471 13419 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:39.678903 13744 consensus_queue.cc:1046] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:39.679798 13744 consensus_queue.cc:1046] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 15:52:39.680441 13744 consensus_queue.cc:1225] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer a53441872c0c416b8e2f26f52399f536 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260809 15:52:39.636520 13628 raft_consensus.cc:2781] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:39.682248 13628 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:39.682648 13628 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:39.639175 13574 raft_consensus.cc:2781] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:39.639840 13243 raft_consensus.cc:3037] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:39.686120 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:39.686925 11636 raft_consensus.cc:2454] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:39.640906 13676 raft_consensus.cc:2781] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:39.671590 10841 raft_consensus.cc:1226] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4. Ops: [3.2-3.2]
I20260809 15:52:39.651451 13504 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:39.658453 13690 raft_consensus.cc:2726] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260809 15:52:39.698210 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 64787.9us
I20260809 15:52:39.639571 13594 consensus_queue.cc:1046] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.052s
I20260809 15:52:39.650413 13588 consensus_queue.cc:1046] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.045s
I20260809 15:52:39.710568 10841 raft_consensus.cc:1226] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:39.693223 13676 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:39.636026 13553 consensus_queue.cc:1046] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.052s
I20260809 15:52:39.695286 13597 raft_consensus.cc:3032] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:39.695497 13653 raft_consensus.cc:2781] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:39.694416 10927 raft_consensus.cc:2454] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:39.719151 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:39.725651 10839 raft_consensus.cc:2396] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260809 15:52:39.725816 13588 raft_consensus.cc:3032] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:39.709648 13702 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:39.719244 10893 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:39.732534 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:39.738265 11015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2209) took 2380 ms. Trace:
W20260809 15:52:39.835662 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 116508us
I20260809 15:52:39.733652 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:52:39.813052 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 113842us
I20260809 15:52:39.749903 11014 raft_consensus.cc:1226] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:39.748625 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1902) took 2382 ms. Trace:
W20260809 15:52:39.837286 10811 leader_election.cc:334] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.938s (SENT)
I20260809 15:52:39.837605 10931 rpcz_store.cc:276] 0809 15:52:37.367438 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.933136 (+1565698us) service_pool.cc:227] Handling call
0809 15:52:38.951814 (+ 18678us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:39.032126 (+ 80312us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:39.748397 (+716271us) spinlock_profiling.cc:243] Waited 79.6 ms on lock 0x7b540025c6d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:39.748566 (+ 169us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":183078144}
I20260809 15:52:39.838186 10893 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:39.838511 10811 leader_election.cc:334] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.992s (SENT)
I20260809 15:52:39.740583 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:39.744485 13522 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:39.840062 13541 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:39.840330 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:39.742695 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:39.739214 11635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 628) took 1811 ms. Trace:
I20260809 15:52:39.730177 10923 raft_consensus.cc:2396] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 15:52:39.842738 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:39.836037 11015 rpcz_store.cc:276] 0809 15:52:37.357679 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.213600 (+855921us) service_pool.cc:227] Handling call
0809 15:52:38.213858 (+ 258us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:38.998989 (+785131us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:39.737982 (+738993us) spinlock_profiling.cc:243] Waited 784 ms on lock 0x7b54000127d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:39.738199 (+ 217us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1804334848}
I20260809 15:52:39.846132 11015 raft_consensus.cc:1226] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:39.847492 11015 raft_consensus.cc:1226] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:39.835963 10840 raft_consensus.cc:2379] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 4: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:39.842156 11635 rpcz_store.cc:276] 0809 15:52:37.927963 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.721072 (+793109us) service_pool.cc:227] Handling call
0809 15:52:38.721359 (+ 287us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:39.713010 (+991651us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:39.713052 (+ 42us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:39.713094 (+ 42us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:52:39.713131 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:39.738962 (+ 25831us) spinlock_profiling.cc:243] Waited 991 ms on lock 0x7b54004d3fd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:39.739017 (+ 55us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:39.739156 (+ 139us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":608,"raft.run_cpu_time_us":131,"raft.run_wall_time_us":5001,"spinlock_wait_cycles":2279853440}
I20260809 15:52:39.835382 10837 raft_consensus.cc:2396] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 15:52:39.856343 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:39.856972 10923 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:39.839057 10811 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:39.713236 13574 raft_consensus.cc:686] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
W20260809 15:52:39.747385 13733 log.cc:924] Time spent T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.521s user 0.004s sys 0.006s
I20260809 15:52:39.867903 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:39.871330 11012 raft_consensus.cc:1226] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:39.719327 13676 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:39.874521 10847 raft_consensus.cc:1226] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:39.875201 10811 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:39.750993 13708 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:39.720956 13597 raft_consensus.cc:729] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:39.888927 10927 raft_consensus.cc:2379] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 2: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:39.740080 13657 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:39.754611 13504 raft_consensus.cc:3032] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:39.836025 13588 raft_consensus.cc:729] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:39.893029 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:39.730298 13677 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:39.856153 13690 raft_consensus.cc:2726] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:39.748381 13509 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:39.813730 11633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 625) took 1986 ms. Trace:
I20260809 15:52:39.867204 13540 raft_consensus.cc:2726] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:39.840651 13541 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:39.910396 10923 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
W20260809 15:52:39.898017 11632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41478 (request call id 945) took 2520 ms (client timeout 1581 ms). Trace:
I20260809 15:52:39.867244 13596 raft_consensus.cc:2726] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:39.867524 13730 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:39.722384 13546 consensus_queue.cc:1046] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.175s
I20260809 15:52:39.925406 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:39.873508 13594 consensus_queue.cc:1035] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
W20260809 15:52:39.929991 10978 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.591s user 0.030s sys 0.033s
I20260809 15:52:39.759290 13628 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:39.911991 13504 raft_consensus.cc:729] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:39.936854 10923 raft_consensus.cc:1261] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260809 15:52:39.903041 13703 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:39.940550 10931 raft_consensus.cc:1261] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260809 15:52:39.942982 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 302389us
W20260809 15:52:39.939806 10978 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.608s user 0.032s sys 0.034s
W20260809 15:52:39.944535 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 745951us
I20260809 15:52:39.947454 10978 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:39.949589 10978 leader_election.cc:302] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:39.950798 10978 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:39.953719 13646 raft_consensus.cc:2781] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:39.954229 13646 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:39.954653 13646 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:39.956941 10978 leader_election.cc:398] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Vote denied by peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783) with higher term. Message: Invalid argument: T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7.
I20260809 15:52:39.957358 10978 leader_election.cc:401] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260809 15:52:39.911103 13571 consensus_queue.cc:1046] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.027s
I20260809 15:52:39.918567 13419 consensus_queue.cc:1046] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.021s
I20260809 15:52:39.958696 10978 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:39.926852 11633 rpcz_store.cc:276] 0809 15:52:37.826348 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.675888 (+849540us) service_pool.cc:227] Handling call
0809 15:52:38.676230 (+ 342us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:39.062826 (+386596us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:39.797895 (+735069us) spinlock_profiling.cc:243] Waited 361 ms on lock 0x7b54005505d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:39.812504 (+ 14609us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":830521728}
I20260809 15:52:39.960079 13689 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 7
I20260809 15:52:39.965607 13702 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:39.959012 13746 raft_consensus.cc:2781] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:39.966616 13746 raft_consensus.cc:482] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:39.967015 13746 raft_consensus.cc:3037] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:39.970795 11009 raft_consensus.cc:1226] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 4. Current term is 5. Ops: []
W20260809 15:52:39.934429 11632 rpcz_store.cc:269] 0809 15:52:37.370732 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:37.658820 (+288088us) service_pool.cc:227] Handling call
0809 15:52:39.874774 (+2215954us) spinlock_profiling.cc:243] Waited 391 ms on lock 0x7b54005d2854. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:39.891144 (+ 16370us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":900175744}
I20260809 15:52:39.933019 13677 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:39.931560 13509 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:39.980299 11636 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41478 (request call id 938) took 2744 ms (client timeout 1937 ms). Trace:
W20260809 15:52:39.980757 11636 rpcz_store.cc:269] 0809 15:52:37.235773 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.133667 (+897894us) service_pool.cc:227] Handling call
0809 15:52:39.980132 (+1846465us) spinlock_profiling.cc:243] Waited 1.52 s on lock 0x7b54004cf754. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:39.980233 (+ 101us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3487861632}
I20260809 15:52:39.981086 13299 raft_consensus.cc:3032] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:39.981562 13628 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:39.958845 13640 raft_consensus.cc:2781] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:39.986264 13640 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:39.986730 13640 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:39.988237 13541 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:39.981585 13299 raft_consensus.cc:729] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:39.990058 13464 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:39.989440 13299 consensus_queue.cc:260] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:39.991231 13299 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:39.992033 13677 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:39.952606 13588 consensus_queue.cc:260] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:39.996951 13588 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:40.007421 13716 raft_consensus.cc:3032] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:40.008026 13716 raft_consensus.cc:729] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:39.941180 13730 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:40.013209 13730 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:40.015527 13703 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:39.939771 13597 consensus_queue.cc:260] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:39.970321 13574 consensus_queue.cc:237] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.027774 13575 raft_consensus.cc:3032] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Stepping down as leader of term 1
W20260809 15:52:40.045113 13722 consensus_meta.cc:287] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.555s user 0.029s sys 0.018s
I20260809 15:52:40.009610 13716 consensus_queue.cc:260] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.057473 13722 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.065593 13722 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:40.071168 13746 raft_consensus.cc:504] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.073973 13746 leader_election.cc:288] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:39.913275 10844 raft_consensus.cc:2396] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 5 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 15:52:39.919198 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:40.109920 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:40.111097 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:40.114099 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 275806us
I20260809 15:52:40.114874 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:40.116333 10842 raft_consensus.cc:2354] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5.
W20260809 15:52:40.119313 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 142652us
I20260809 15:52:40.120692 10923 raft_consensus.cc:1226] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
W20260809 15:52:40.122633 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 493676us
I20260809 15:52:40.124199 11606 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:40.125352 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 166581us
I20260809 15:52:40.125873 13546 raft_consensus.cc:2726] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 15:52:40.122663 10930 raft_consensus.cc:2454] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:40.128592 13594 consensus_queue.cc:1046] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:40.129824 11632 raft_consensus.cc:3032] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:40.130349 11010 raft_consensus.cc:1226] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:40.131232 13594 raft_consensus.cc:3032] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:40.131773 13594 raft_consensus.cc:729] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:40.031720 13504 consensus_queue.cc:260] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.133370 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:40.134742 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 287668us
I20260809 15:52:40.134384 13594 consensus_queue.cc:260] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.135468 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:40.136991 13594 raft_consensus.cc:3037] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:40.137152 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:40.108011 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 204157us
W20260809 15:52:40.138330 13465 consensus_meta.cc:287] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.586s user 0.021s sys 0.000s
I20260809 15:52:40.138672 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:40.139710 10838 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:40.140437 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:40.141731 11632 raft_consensus.cc:729] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:40.143154 13509 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:40.138723 10842 raft_consensus.cc:2396] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for term 4 because replica has last-logged OpId of term: 3 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 15:52:40.043923 13575 raft_consensus.cc:729] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:40.146055 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:40.147104 11018 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:40.148553 13575 consensus_queue.cc:260] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.139240 10811 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:40.150847 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:40.152694 13730 raft_consensus.cc:2781] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:40.153204 13730 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:40.143604 11632 consensus_queue.cc:260] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.153577 13730 raft_consensus.cc:3037] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:40.154629 11632 raft_consensus.cc:3037] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:40.139185 13465 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:40.157683 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:40.206427 13464 consensus_queue.cc:1046] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:40.207615 13575 raft_consensus.cc:3037] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:40.210599 10848 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:40.214795 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:40.216037 10837 raft_consensus.cc:2379] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 4: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:40.220098 13746 raft_consensus.cc:2781] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election won for term 3
W20260809 15:52:40.197223 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 76388.6us
I20260809 15:52:40.227471 13653 raft_consensus.cc:686] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:40.227694 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:52:40.227995 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 104078us
I20260809 15:52:40.228835 10837 raft_consensus.cc:2379] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 4: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:40.019815 13646 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.233284 13646 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:40.239403 13473 consensus_queue.cc:1046] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:40.240070 13509 raft_consensus.cc:2726] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 15:52:40.240926 13509 raft_consensus.cc:3032] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:40.241330 13509 raft_consensus.cc:729] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:40.243486 13465 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:40.247097 13716 raft_consensus.cc:3037] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:40.247193 10926 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:40.220737 13746 raft_consensus.cc:686] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:40.249476 13746 consensus_queue.cc:237] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.250746 13423 consensus_queue.cc:1046] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:40.255544 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:40.226643 10838 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:40.271960 13629 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:40.272930 13629 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.275705 11636 raft_consensus.cc:1226] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260809 15:52:40.276384 10921 raft_consensus.cc:3032] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:40.276806 10921 raft_consensus.cc:729] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:40.278196 10921 consensus_queue.cc:260] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:40.275009 13509 consensus_queue.cc:260] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.281368 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:40.277669 13465 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:40.229257 13653 consensus_queue.cc:237] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.287400 10922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1903) took 2918 ms. Trace:
W20260809 15:52:40.288348 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 59258.3us
I20260809 15:52:40.289161 11018 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:40.295078 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:52:40.298259 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 48099.5us
I20260809 15:52:40.280725 13730 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.312958 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:40.313800 11009 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:40.314574 11013 raft_consensus.cc:2379] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 6: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:40.303764 10921 raft_consensus.cc:3037] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:40.296489 13437 raft_consensus.cc:2726] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:40.320624 10838 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:40.325984 13464 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:40.307976 13473 consensus_queue.cc:1046] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.020s
I20260809 15:52:40.328624 11632 raft_consensus.cc:1261] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 5. (term mismatch)
I20260809 15:52:40.289907 10922 rpcz_store.cc:276] 0809 15:52:37.367780 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.935636 (+1567856us) service_pool.cc:227] Handling call
0809 15:52:38.963486 (+ 27850us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:39.007186 (+ 43700us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:39.009137 (+ 1951us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:39.507509 (+498372us) spinlock_profiling.cc:243] Waited 3.68 ms on lock 0x7b2c0000f228. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7967ed6e11 00007f7967ee6442 00007f796952ec11 00007f79694f358a 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772
0809 15:52:39.507983 (+ 474us) log.cc:842] Serialized 26 byte log entry
0809 15:52:39.832869 (+324886us) raft_consensus.cc:1631] Marking committed up to 3
0809 15:52:40.258508 (+425639us) log.cc:842] Serialized 14 byte log entry
0809 15:52:40.285098 (+ 26590us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:40.285269 (+ 171us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:40.285345 (+ 76us) raft_consensus.cc:1671] finished
0809 15:52:40.285477 (+ 132us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:40.285685 (+ 208us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":12898,"prepare.queue_time_us":378129,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":59,"raft.queue_time_us":12743,"raft.run_cpu_time_us":340,"raft.run_wall_time_us":12071,"spinlock_wait_cycles":8458624,"thread_start_us":645184,"threads_started":2,"wal-append.queue_time_us":447708}
I20260809 15:52:40.334611 13629 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:40.335945 13465 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:40.336907 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:40.337132 13723 raft_consensus.cc:3032] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:40.338101 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
W20260809 15:52:40.338665 10930 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54224 (request call id 1900) took 2973 ms (client timeout 2316 ms). Trace:
I20260809 15:52:40.342716 13597 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:40.346112 13299 consensus_queue.cc:1046] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260809 15:52:40.347148 13757 consensus_queue.cc:1046] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 15:52:40.373203 13730 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:40.369199 13723 raft_consensus.cc:729] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:40.347193 11606 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:40.348387 13504 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:40.377350 13723 consensus_queue.cc:260] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:40.378825 13723 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:40.370878 11636 raft_consensus.cc:2454] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:40.384389 13473 consensus_queue.cc:1046] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260809 15:52:40.385437 13299 consensus_queue.cc:1225] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer a53441872c0c416b8e2f26f52399f536 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260809 15:52:40.386127 13299 raft_consensus.cc:3032] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:40.386576 13299 raft_consensus.cc:729] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
W20260809 15:52:40.388530 13768 log.cc:924] Time spent T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.322s user 0.002s sys 0.000s
I20260809 15:52:40.389129 13757 consensus_queue.cc:1225] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer a53441872c0c416b8e2f26f52399f536 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260809 15:52:40.389863 13757 raft_consensus.cc:3032] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:40.390228 13473 raft_consensus.cc:3032] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:52:40.390306 13757 raft_consensus.cc:729] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:40.390826 13473 raft_consensus.cc:729] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:40.391876 13473 consensus_queue.cc:260] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.392783 13628 raft_consensus.cc:3032] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:40.393426 13473 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:40.357407 13464 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.393334 13628 raft_consensus.cc:729] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:40.395692 13464 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:40.349529 10893 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:40.367158 11009 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:40.370676 11633 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
W20260809 15:52:40.369688 10930 rpcz_store.cc:269] 0809 15:52:37.365516 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:38.085680 (+720164us) service_pool.cc:227] Handling call
0809 15:52:40.338382 (+2252702us) spinlock_profiling.cc:243] Waited 2.04 s on lock 0x7b540021b1d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:40.338570 (+ 188us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4681792384}
I20260809 15:52:40.371863 13522 raft_consensus.cc:482] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:40.387812 13299 consensus_queue.cc:260] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:40.366744 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 51711.3us
I20260809 15:52:40.393280 10737 catalog_manager.cc:5495] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 reported cstate change: config changed from index 2 to 3, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New cstate: current_term: 1 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:40.362516 11632 pending_rounds.cc:76] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9: Aborting all ops after (but not including) 4
I20260809 15:52:40.406736 11632 pending_rounds.cc:97] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9: Aborting uncommitted NO_OP operation due to leader change: 2.5
W20260809 15:52:40.406175 10811 leader_election.cc:334] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.540s (SENT)
I20260809 15:52:40.411345 10838 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:40.410471 10840 raft_consensus.cc:2454] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:40.418676 10811 leader_election.cc:302] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:40.407140 13509 raft_consensus.cc:3037] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:40.413481 11632 raft_consensus.cc:2866] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:40.425410 13656 raft_consensus.cc:2726] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:40.425675 13464 raft_consensus.cc:2726] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:40.431268 13334 raft_consensus.cc:3032] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 15:52:40.460659 13334 raft_consensus.cc:729] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:40.462262 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:40.433780 10924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 714) took 1012 ms. Trace:
I20260809 15:52:40.436952 13594 raft_consensus.cc:2726] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:40.440387 10737 catalog_manager.cc:5495] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 3 to 5, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 5 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:40.442173 13522 raft_consensus.cc:504] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.461776 13334 consensus_queue.cc:260] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:40.436072 10811 leader_election.cc:334] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.624s (SENT)
W20260809 15:52:40.469939 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 120522us
I20260809 15:52:40.471886 13334 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:52:40.473731 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:40.474026 11009 raft_consensus.cc:2379] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 6: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:40.391660 13757 consensus_queue.cc:260] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:40.475165 10811 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.628s (SENT) [suppressed 56 similar messages]
I20260809 15:52:40.474985 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:40.477634 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 130594us
W20260809 15:52:40.475749 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 56108.7us
I20260809 15:52:40.485760 10848 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:40.395818 13628 consensus_queue.cc:260] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.477880 13299 raft_consensus.cc:3037] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:40.463209 10924 rpcz_store.cc:276] 0809 15:52:39.403923 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:39.532317 (+128394us) service_pool.cc:227] Handling call
0809 15:52:39.572992 (+ 40675us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:39.573233 (+ 241us) raft_consensus.cc:1520] Early marking committed up to index 6
0809 15:52:39.943912 (+370679us) log.cc:842] Serialized 14 byte log entry
0809 15:52:40.308744 (+364832us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:40.352621 (+ 43877us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:40.369623 (+ 17002us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:40.404943 (+ 35320us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:40.416216 (+ 11273us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1456,"prepare.queue_time_us":495848,"prepare.run_cpu_time_us":97,"prepare.run_wall_time_us":1637,"raft.queue_time_us":11361,"raft.run_cpu_time_us":224,"raft.run_wall_time_us":11749,"thread_start_us":639699,"threads_started":2}
I20260809 15:52:40.495944 11013 raft_consensus.cc:1226] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:40.498919 13503 consensus_queue.cc:1046] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260809 15:52:40.498811 13640 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:40.517166 13473 consensus_queue.cc:1046] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 15:52:40.518046 10920 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 708) took 1173 ms. Trace:
I20260809 15:52:40.518627 13744 raft_consensus.cc:3032] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:40.520053 11013 raft_consensus.cc:1226] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:40.520635 11009 raft_consensus.cc:1261] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 15:52:40.523598 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:40.525354 13522 leader_election.cc:288] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:40.527148 10931 raft_consensus.cc:1261] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260809 15:52:40.531370 11012 raft_consensus.cc:2379] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 6: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:40.514361 10737 catalog_manager.cc:5495] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 3 to 5, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 5 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:40.530350 11606 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:40.536255 11018 raft_consensus.cc:2379] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 4: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:40.536336 13473 raft_consensus.cc:3032] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:40.536788 13473 raft_consensus.cc:729] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:40.530167 13744 raft_consensus.cc:729] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:40.537810 13473 consensus_queue.cc:260] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.539426 13473 raft_consensus.cc:3037] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:40.538028 13744 consensus_queue.cc:260] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.529680 13758 raft_consensus.cc:2726] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:40.544018 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:40.544428 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:40.544762 11009 raft_consensus.cc:2379] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 3: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
W20260809 15:52:40.546975 13708 consensus_meta.cc:287] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.649s user 0.013s sys 0.000s
I20260809 15:52:40.547786 10924 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:40.548882 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:40.541842 10737 catalog_manager.cc:5495] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 3 to 5, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 5 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:40.531723 13757 raft_consensus.cc:3037] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:40.542219 13628 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
W20260809 15:52:40.555033 13676 consensus_meta.cc:287] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.643s user 0.010s sys 0.003s
W20260809 15:52:40.557190 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 49948.1us
I20260809 15:52:40.556885 10737 catalog_manager.cc:5495] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 4 to 6, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 6 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:40.553464 13465 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:40.584641 10922 raft_consensus.cc:1261] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:40.581449 10893 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:40.574520 13597 raft_consensus.cc:2726] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 15:52:40.591658 11606 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:40.592229 10737 catalog_manager.cc:5495] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 3 to 4, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 4 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:40.606475 13744 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:40.607335 10737 catalog_manager.cc:5495] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 1 to 3, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 3 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:40.609141 10737 catalog_manager.cc:5495] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 2 to 4, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 4 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:40.610229 13574 consensus_queue.cc:1035] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.028s
I20260809 15:52:40.612707 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:40.613416 11018 raft_consensus.cc:2454] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:40.611354 10737 catalog_manager.cc:5495] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 4 to 6, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 6 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
W20260809 15:52:40.595556 10978 connection.cc:232] RPC call timeout handler was delayed by -1.5178744229488075s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20920000000000002s for a response
W20260809 15:52:40.618002 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 142890us
I20260809 15:52:40.586022 11633 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:40.626564 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:40.579993 13676 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.627888 10737 catalog_manager.cc:5495] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 2 to 3, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 3 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:40.635174 11606 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:40.590055 13653 consensus_queue.cc:1046] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.037s
W20260809 15:52:40.637919 10978 leader_election.cc:334] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.524s (SENT)
I20260809 15:52:40.639766 11017 raft_consensus.cc:1261] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260809 15:52:40.641996 13597 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:40.642668 13597 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.644999 13597 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:40.640544 11018 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
W20260809 15:52:40.647732 10811 leader_election.cc:334] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.709s (SENT)
I20260809 15:52:40.622306 13629 raft_consensus.cc:2726] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 15:52:40.639725 10737 catalog_manager.cc:5495] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 1 to 3, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 3 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:40.660681 13640 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:40.661629 10737 catalog_manager.cc:5495] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 1 to 3, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 3 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:40.662400 13597 consensus_queue.cc:1046] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
W20260809 15:52:40.663141 13785 log.cc:924] Time spent T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.137s user 0.001s sys 0.003s
I20260809 15:52:40.663312 13597 consensus_queue.cc:1225] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer a53441872c0c416b8e2f26f52399f536 log is divergent from this leader: its last log entry 7.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260809 15:52:40.664119 13503 raft_consensus.cc:3032] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:52:40.664593 13503 raft_consensus.cc:729] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
W20260809 15:52:40.665337 13689 consensus_meta.cc:287] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.701s user 0.014s sys 0.001s
I20260809 15:52:40.663811 10737 catalog_manager.cc:5495] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 reported cstate change: term changed from 4 to 6, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to c7aa762318cb4e97875333f10679a9f9 (127.9.84.4), now has a pending config: VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1), VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2), VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 6 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } } pending_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:40.665856 13689 raft_consensus.cc:2726] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783) with higher term. Message: Invalid argument: T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7.
I20260809 15:52:40.665601 13503 consensus_queue.cc:260] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:40.651607 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 99643.5us
I20260809 15:52:40.530983 10920 rpcz_store.cc:276] 0809 15:52:39.337240 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:39.460532 (+123292us) service_pool.cc:227] Handling call
0809 15:52:40.360533 (+900001us) spinlock_profiling.cc:243] Waited 49.4 ms on lock 0x7b58000d1498. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f7966593851 00007f7966593702 00007f796f483b09 00007f796e3bf730 00007f796e48579b 00007f796e485821 00007f796e4492b0 00007f796e4347f9 00007f796e4342a2 00007f7967b1a684
0809 15:52:40.360905 (+ 372us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:40.361362 (+ 457us) raft_consensus.cc:1520] Early marking committed up to index 6
0809 15:52:40.361789 (+ 427us) log.cc:842] Serialized 14 byte log entry
0809 15:52:40.487728 (+125939us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:40.489485 (+ 1757us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:40.494560 (+ 5075us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:40.497508 (+ 2948us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:40.508262 (+ 10754us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":1190,"prepare.run_cpu_time_us":72,"prepare.run_wall_time_us":2241,"raft.queue_time_us":11341,"raft.run_cpu_time_us":110,"raft.run_wall_time_us":3002,"spinlock_wait_cycles":117382656,"thread_start_us":106936,"threads_started":1,"wal-append.queue_time_us":163416}
I20260809 15:52:40.658272 13513 raft_consensus.cc:2726] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260809 15:52:40.658772 13758 raft_consensus.cc:2781] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:40.668458 13758 raft_consensus.cc:686] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:40.670881 13503 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
W20260809 15:52:40.667347 13782 log.cc:924] Time spent T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.142s user 0.002s sys 0.000s
I20260809 15:52:40.671799 11606 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:40.649791 10978 leader_election.cc:302] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:40.677040 13597 raft_consensus.cc:2726] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260809 15:52:40.677731 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 46702.5us
W20260809 15:52:40.678488 10978 leader_election.cc:334] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Timed out: RequestConsensusVote RPC to 127.9.84.2:32783 timed out after 1.565s (SENT)
I20260809 15:52:40.678823 10978 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:40.679296 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:40.679524 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:40.679571 13640 raft_consensus.cc:2726] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:40.680197 11633 raft_consensus.cc:2379] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 5: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:40.680124 13689 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:40.680291 11632 raft_consensus.cc:2454] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
W20260809 15:52:40.686689 10978 leader_election.cc:334] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.580s (SENT)
I20260809 15:52:40.689183 10811 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:40.669400 13758 consensus_queue.cc:237] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.697400 10893 leader_election.cc:398] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Vote denied by peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) with higher term. Message: Invalid argument: T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5.
I20260809 15:52:40.697741 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:40.709564 10893 leader_election.cc:401] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260809 15:52:40.699468 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:40.709993 10921 raft_consensus.cc:1261] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260809 15:52:40.710433 11640 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
W20260809 15:52:40.710785 13344 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260809 15:52:40.700455 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:40.710934 10893 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:40.711649 11639 raft_consensus.cc:2379] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 3: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:40.712216 11633 raft_consensus.cc:2379] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 4: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:40.712425 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:40.700524 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:40.701639 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:40.701630 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:40.714161 11636 raft_consensus.cc:2379] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 2: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:40.714061 10893 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:40.714407 13650 raft_consensus.cc:2726] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) with higher term. Message: Invalid argument: T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5.
I20260809 15:52:40.715699 13708 raft_consensus.cc:2726] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:40.716639 13708 raft_consensus.cc:2726] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:40.716544 13650 consensus_queue.cc:1046] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:40.717661 13650 raft_consensus.cc:3032] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:40.718027 13650 raft_consensus.cc:729] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:40.702243 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:40.718935 13650 consensus_queue.cc:260] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:40.719826 11635 raft_consensus.cc:2454] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:52:40.719976 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:40.718147 13657 consensus_meta.cc:287] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.673s user 0.010s sys 0.006s
I20260809 15:52:40.720625 11018 raft_consensus.cc:2454] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:40.721581 13762 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:40.722123 13762 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.723649 13650 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:40.727106 10927 raft_consensus.cc:1226] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260809 15:52:40.729588 11604 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:40.729807 13522 raft_consensus.cc:3032] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:40.730321 13522 raft_consensus.cc:729] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:40.731197 10893 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:40.731531 13522 consensus_queue.cc:260] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.733566 13522 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:40.738268 10927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1930) took 1095 ms. Trace:
I20260809 15:52:40.734741 13708 raft_consensus.cc:2726] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:40.738652 10927 rpcz_store.cc:276] 0809 15:52:39.643326 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:39.938634 (+295308us) service_pool.cc:227] Handling call
0809 15:52:40.109696 (+171062us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:40.727546 (+617850us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:40.738001 (+ 10455us) spinlock_profiling.cc:243] Waited 617 ms on lock 0x7b54000a91d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:40.738212 (+ 211us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1418908928}
I20260809 15:52:40.743991 13762 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:40.745434 13676 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:40.745296 13344 consensus_queue.cc:1046] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
W20260809 15:52:40.746747 13781 log.cc:924] Time spent T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.222s user 0.000s sys 0.003s
I20260809 15:52:40.746848 13344 raft_consensus.cc:3032] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:40.747731 13344 raft_consensus.cc:729] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:40.748705 13344 consensus_queue.cc:260] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:40.754380 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:40.753159 11642 raft_consensus.cc:2454] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:40.755033 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:40.756891 11014 raft_consensus.cc:2454] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:40.757743 11635 raft_consensus.cc:2454] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:40.754346 13344 raft_consensus.cc:3037] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:40.759418 13716 raft_consensus.cc:2726] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:40.760284 10809 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:40.760730 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:40.761313 13716 raft_consensus.cc:2726] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:40.762359 10931 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:40.763011 11604 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:40.764004 13723 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:40.772056 11637 raft_consensus.cc:2454] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:40.775291 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:40.775339 11639 raft_consensus.cc:2454] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:40.778708 10893 leader_election.cc:302] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:40.784175 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 151578us
I20260809 15:52:40.787788 11636 raft_consensus.cc:1226] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 8. Current term is 9. Ops: [8.9-8.9]
I20260809 15:52:40.786906 11642 raft_consensus.cc:1261] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 5 LEARNER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260809 15:52:40.770509 10921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1922) took 1801 ms. Trace:
I20260809 15:52:40.789836 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:40.777670 13762 raft_consensus.cc:2781] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:40.777289 13708 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:40.798660 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:40.766630 13723 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:40.804279 13334 raft_consensus.cc:2726] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
W20260809 15:52:40.805153 13772 log.cc:924] Time spent T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.280s user 0.005s sys 0.001s
I20260809 15:52:40.805759 13762 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:40.805869 13435 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:40.800911 11633 raft_consensus.cc:2396] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 8 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260809 15:52:40.817741 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:40.809937 13762 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:40.822364 11640 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:40.830288 10893 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:40.828608 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:40.831337 11636 raft_consensus.cc:2362] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4. Re-sending same reply.
I20260809 15:52:40.827615 13464 consensus_queue.cc:1046] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 15:52:40.839999 13723 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:40.843689 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 79077.5us
I20260809 15:52:40.842341 10893 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:40.797881 10921 rpcz_store.cc:276] 0809 15:52:38.969537 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:39.438272 (+468735us) service_pool.cc:227] Handling call
0809 15:52:39.461952 (+ 23680us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:40.275812 (+813860us) spinlock_profiling.cc:243] Waited 56.3 ms on lock 0x7b5400444920. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f7969483730 00007f79694f5300 00007f79694f6371 00007f79694f2edf 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc
0809 15:52:40.710447 (+434635us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:40.766978 (+ 56531us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":129416960}
I20260809 15:52:40.844550 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:40.855686 11014 raft_consensus.cc:2454] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:40.856462 10891 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:40.857698 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:40.819082 13708 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.856993 13334 raft_consensus.cc:3032] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:40.859688 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:40.859974 13653 raft_consensus.cc:2781] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:40.821662 13435 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.864923 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:40.859776 13629 consensus_queue.cc:1035] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:40.862190 13334 raft_consensus.cc:729] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:40.841673 13676 raft_consensus.cc:2781] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:40.842984 13689 raft_consensus.cc:2726] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260809 15:52:40.867002 10811 leader_election.cc:334] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 2.259s (SENT)
W20260809 15:52:40.871099 10978 leader_election.cc:334] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.634s (SENT)
I20260809 15:52:40.873728 10811 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:40.867883 13464 raft_consensus.cc:2781] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:40.879274 13464 raft_consensus.cc:686] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:40.880585 10921 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:40.880517 13464 consensus_queue.cc:237] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.863986 13653 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:40.883599 10978 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:40.884981 13679 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:40.885514 13679 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.887581 13679 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:40.894635 13716 raft_consensus.cc:2726] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:40.894891 11636 raft_consensus.cc:2454] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:40.898556 13708 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:40.896526 13653 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:40.905701 11606 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:40.908905 10809 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:40.909770 13716 raft_consensus.cc:2781] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:40.910377 13716 raft_consensus.cc:686] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:40.911844 13716 consensus_queue.cc:237] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:40.921041 13435 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:40.917922 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 51020.1us
I20260809 15:52:40.924270 10811 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:40.925143 13594 raft_consensus.cc:2726] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:52:40.925580 10811 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:40.920303 13334 consensus_queue.cc:260] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.931836 10809 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:40.870287 13676 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 15:52:40.954314 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 70928.6us
I20260809 15:52:40.955243 10893 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:40.957582 13629 raft_consensus.cc:2781] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:40.958137 13629 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:40.958475 13594 raft_consensus.cc:2726] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:40.958489 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:40.958909 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:40.953580 13504 raft_consensus.cc:2726] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:40.953733 13640 raft_consensus.cc:2726] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 15:52:40.959771 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:40.961033 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:40.940339 13762 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.941576 11640 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:40.868295 11632 raft_consensus.cc:2454] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:40.962546 10922 raft_consensus.cc:2396] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260809 15:52:40.965029 11635 raft_consensus.cc:2454] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:40.968739 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:40.971578 13762 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:40.972566 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:40.972838 13299 raft_consensus.cc:2781] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:40.973181 11013 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:40.956153 13676 raft_consensus.cc:3037] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:40.958495 13629 raft_consensus.cc:3037] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:40.991814 11013 raft_consensus.cc:2454] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:40.992971 11604 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:40.993737 13762 raft_consensus.cc:2781] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:40.994274 13762 raft_consensus.cc:686] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:40.995218 13762 consensus_queue.cc:237] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:40.965744 13628 raft_consensus.cc:2741] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260809 15:52:41.007148 10920 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:40.974020 13299 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:41.009181 13299 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:41.010180 13334 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:41.014711 13653 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:41.017066 13653 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:41.018151 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:41.018792 11013 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:41.019040 10927 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
W20260809 15:52:41.022394 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 157920us
W20260809 15:52:41.023620 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 55435.4us
I20260809 15:52:41.025154 11013 raft_consensus.cc:2454] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:41.026309 11606 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:41.025750 10930 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:41.027788 10891 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:41.030189 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:41.031117 10809 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:41.033494 10811 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:41.036746 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:41.031160 13653 raft_consensus.cc:2781] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:41.033623 10921 raft_consensus.cc:595] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:41.047748 10809 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:41.049007 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:41.027426 13504 consensus_queue.cc:1046] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.013s
W20260809 15:52:41.086767 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 109806us
W20260809 15:52:41.089788 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 59890.2us
I20260809 15:52:41.047101 13656 raft_consensus.cc:2781] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:41.044850 13628 raft_consensus.cc:2726] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:41.047195 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:41.099680 13588 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:41.102936 13588 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.099917 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:41.100948 13435 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:41.101579 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:41.107391 10930 raft_consensus.cc:3037] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:41.049256 13594 consensus_queue.cc:1035] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260809 15:52:41.092608 13653 raft_consensus.cc:686] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:41.116338 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:41.120924 10893 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:41.118677 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:41.094954 13716 raft_consensus.cc:2726] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:41.095309 13656 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:41.128875 11640 raft_consensus.cc:2454] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
W20260809 15:52:41.135043 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 91461us
I20260809 15:52:41.138561 10848 raft_consensus.cc:1226] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:41.138365 10893 leader_election.cc:302] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:41.138283 13437 raft_consensus.cc:2781] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:41.139130 10838 raft_consensus.cc:1226] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
W20260809 15:52:41.137720 10978 leader_election.cc:334] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.700s (SENT)
I20260809 15:52:41.139273 13437 raft_consensus.cc:686] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:41.142333 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:41.150466 10840 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:41.150972 10839 raft_consensus.cc:1261] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:41.150800 13437 consensus_queue.cc:237] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.143777 10848 raft_consensus.cc:1226] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:41.144017 13575 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 15:52:41.144826 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:41.153146 10838 raft_consensus.cc:2379] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 3: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:41.145403 13653 consensus_queue.cc:237] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:41.147164 10837 raft_consensus.cc:1261] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 3. (term mismatch)
I20260809 15:52:41.154106 10837 pending_rounds.cc:76] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536: Aborting all ops after (but not including) 2
I20260809 15:52:41.154333 10837 pending_rounds.cc:97] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536: Aborting uncommitted NO_OP operation due to leader change: 3.3
I20260809 15:52:41.154562 10837 raft_consensus.cc:2866] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:41.148787 10843 raft_consensus.cc:1226] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 8. Current term is 9. Ops: []
I20260809 15:52:41.149573 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:41.166146 10930 raft_consensus.cc:2454] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:41.113481 13503 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:41.171691 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 77675.3us
I20260809 15:52:41.141016 13656 raft_consensus.cc:3037] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:41.178653 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:41.178231 10893 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
W20260809 15:52:41.181437 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 72553.8us
I20260809 15:52:41.157860 13716 raft_consensus.cc:2781] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:41.113493 13435 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.156802 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:41.185395 10811 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:41.170631 13708 raft_consensus.cc:2741] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260809 15:52:41.194463 13464 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:41.194612 13650 raft_consensus.cc:2781] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:41.197186 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:41.199357 13800 log.cc:924] Time spent T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.059s user 0.000s sys 0.000s
W20260809 15:52:41.203622 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 96554.3us
I20260809 15:52:41.170832 13640 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
W20260809 15:52:41.181241 13801 log.cc:924] Time spent T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.077s user 0.001s sys 0.001s
I20260809 15:52:41.206158 10843 raft_consensus.cc:1261] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 3. (term mismatch)
I20260809 15:52:41.214111 10843 pending_rounds.cc:76] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536: Aborting all ops after (but not including) 2
I20260809 15:52:41.214350 10843 pending_rounds.cc:97] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536: Aborting uncommitted NO_OP operation due to leader change: 4.3
I20260809 15:52:41.214565 10843 raft_consensus.cc:2866] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:41.211485 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:41.184454 13716 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:41.215024 13575 raft_consensus.cc:2741] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260809 15:52:41.222446 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:41.229990 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 51228.6us
I20260809 15:52:41.230844 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:41.208845 13650 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:41.238723 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:41.239687 13473 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:41.240712 11013 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:41.228489 13716 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:41.244144 10891 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:41.190745 13503 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.236892 13588 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:41.236748 13640 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:41.246948 13473 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.207142 11632 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:41.248041 10840 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:41.252593 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:41.253132 13344 raft_consensus.cc:2781] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260809 15:52:41.259024 10978 leader_election.cc:334] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.820s (SENT)
I20260809 15:52:41.260524 13435 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:41.215101 13504 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260809 15:52:41.266026 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:41.277381 10978 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:41.266670 13503 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:41.238216 13650 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:41.269492 11014 raft_consensus.cc:1226] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:41.222615 13464 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:41.284102 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 87394.8us
I20260809 15:52:41.223897 13629 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.298904 10811 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:41.276357 13344 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:41.301726 11606 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:41.303746 10927 raft_consensus.cc:2379] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 4: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:41.306427 13503 raft_consensus.cc:2726] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:41.297442 13689 raft_consensus.cc:2726] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:41.308709 13575 consensus_queue.cc:1035] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260809 15:52:41.314838 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:41.303100 13344 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
W20260809 15:52:41.318212 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 59274.2us
I20260809 15:52:41.323123 11632 raft_consensus.cc:1261] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260809 15:52:41.323276 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:41.323971 11639 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:41.324748 13473 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:41.290200 13676 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.327864 13813 raft_consensus.cc:2741] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260809 15:52:41.331483 13594 consensus_queue.cc:1046] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:41.345736 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:41.331729 13810 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:41.332029 13629 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:41.332145 13808 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:41.333315 13504 consensus_queue.cc:1046] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:41.329253 13653 consensus_queue.cc:1035] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.006s
I20260809 15:52:41.336998 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:41.363251 10837 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
W20260809 15:52:41.354499 10978 leader_election.cc:334] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.937s (SENT)
I20260809 15:52:41.354604 11013 raft_consensus.cc:2379] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 4: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:41.356560 11606 leader_election.cc:398] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Vote denied by peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783) with higher term. Message: Invalid argument: T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3.
I20260809 15:52:41.305291 13806 consensus_queue.cc:1035] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260809 15:52:41.363826 10978 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:41.355048 13522 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:41.368196 10893 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:41.366350 13808 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:41.369938 13628 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:41.370481 13628 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.371443 13798 raft_consensus.cc:2726] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:52:41.372330 13809 raft_consensus.cc:3032] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:41.372834 13809 raft_consensus.cc:729] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:41.372442 13628 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:41.373652 13809 consensus_queue.cc:260] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:41.375365 13809 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
W20260809 15:52:41.376062 10978 leader_election.cc:334] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 2.147s (SENT)
I20260809 15:52:41.357157 13810 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:41.377204 10978 connection.cc:232] RPC call timeout handler was delayed by -1.081500712025445s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.24480000000000002s for a response
I20260809 15:52:41.366683 13522 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.378258 13689 raft_consensus.cc:2726] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:41.379819 13522 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:41.379892 13810 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:41.380625 13810 consensus_queue.cc:1035] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:41.383145 13813 raft_consensus.cc:3032] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:41.383596 13813 raft_consensus.cc:729] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:41.384392 13813 consensus_queue.cc:260] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.385388 13813 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:41.385453 10893 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:41.387104 13653 consensus_queue.cc:1035] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:52:41.387696 13679 raft_consensus.cc:2726] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:41.373230 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:41.371533 13808 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:41.398203 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:41.405748 10893 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:41.400429 13716 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:41.408571 10978 leader_election.cc:398] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Vote denied by peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) with higher term. Message: Invalid argument: T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7.
I20260809 15:52:41.409710 13716 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:41.410043 10978 leader_election.cc:398] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Vote denied by peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) with higher term. Message: Invalid argument: T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5.
I20260809 15:52:41.410647 13640 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:41.414965 10893 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:41.358603 13299 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.367244 13629 consensus_queue.cc:1035] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.039s
I20260809 15:52:41.428499 10978 leader_election.cc:401] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Cancelling election due to peer responding with higher term
W20260809 15:52:41.423882 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 65666.3us
W20260809 15:52:41.428319 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 54734.8us
I20260809 15:52:41.430876 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:41.431630 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:41.431525 11013 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:41.423889 13798 raft_consensus.cc:2781] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:41.435674 13798 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:41.436085 13798 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:41.436076 11016 raft_consensus.cc:1226] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:41.444845 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:41.445076 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:41.445755 11009 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:41.446166 13723 raft_consensus.cc:2726] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:41.446802 10927 raft_consensus.cc:1261] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:41.447160 13464 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:41.449077 10809 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:41.452004 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:41.453011 11013 raft_consensus.cc:2379] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 3: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:41.445756 10921 raft_consensus.cc:2379] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 3: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:41.451525 10837 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:41.459777 10978 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:41.460645 10978 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:41.460465 13798 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.461530 10978 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:41.462546 13798 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:41.462709 13594 raft_consensus.cc:2781] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:41.463101 13594 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:41.463387 10927 raft_consensus.cc:1261] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:41.465945 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:41.466271 10837 raft_consensus.cc:1261] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260809 15:52:41.468343 11009 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:41.469719 10809 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:41.473423 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:41.474725 10930 raft_consensus.cc:1226] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:41.475135 10840 raft_consensus.cc:2454] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:41.475476 10978 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:41.453964 13640 raft_consensus.cc:2726] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election lost for term 4. Reason: Vote denied by peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) with higher term. Message: Invalid argument: T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5.
I20260809 15:52:41.478302 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:41.478281 10893 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:41.480188 10848 raft_consensus.cc:1261] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260809 15:52:41.477022 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:41.463425 13594 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:41.482357 13708 raft_consensus.cc:2781] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260809 15:52:41.482648 10811 leader_election.cc:334] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.718s (SENT)
I20260809 15:52:41.482861 13806 raft_consensus.cc:3032] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:41.482764 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:41.485347 10922 raft_consensus.cc:1226] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 6. Current term is 7. Ops: [6.8-6.8]
I20260809 15:52:41.486620 13575 raft_consensus.cc:2781] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:41.476677 13820 raft_consensus.cc:2781] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:52:41.486675 13656 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.490257 13676 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:41.494336 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:41.496696 13299 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:41.500032 11636 raft_consensus.cc:1261] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:41.500576 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:41.500939 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:41.501267 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:41.516521 11637 raft_consensus.cc:3037] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:41.502200 11633 raft_consensus.cc:1261] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260809 15:52:41.505014 10843 raft_consensus.cc:2379] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 3: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:41.506243 11635 raft_consensus.cc:2454] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:41.448297 13653 consensus_queue.cc:1035] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.028s
I20260809 15:52:41.507038 11640 raft_consensus.cc:2454] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:41.471776 13689 raft_consensus.cc:2781] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:41.513882 13640 raft_consensus.cc:2781] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:41.532559 13689 raft_consensus.cc:686] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:41.509152 13806 raft_consensus.cc:729] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:41.475162 13819 raft_consensus.cc:2781] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:41.516096 11639 raft_consensus.cc:2454] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:41.461993 13650 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.520200 10930 raft_consensus.cc:1226] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 5. Current term is 7. Ops: []
I20260809 15:52:41.538985 11637 raft_consensus.cc:2454] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:41.512056 13820 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:41.520797 10893 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:41.521904 13676 consensus_queue.cc:1035] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:41.523597 10840 raft_consensus.cc:1226] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:41.526520 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:41.506613 13708 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:41.509752 13435 consensus_queue.cc:1035] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.021s
I20260809 15:52:41.515367 13575 raft_consensus.cc:686] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
W20260809 15:52:41.531147 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 48559.3us
I20260809 15:52:41.531901 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
W20260809 15:52:41.538965 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 65997.6us
I20260809 15:52:41.555670 10924 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:41.501734 13344 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.544894 10931 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:41.565075 10893 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:41.535468 11632 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:41.569556 10922 raft_consensus.cc:1226] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260809 15:52:41.570547 10893 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:41.556152 10921 raft_consensus.cc:2379] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 3: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:41.577690 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:41.578500 11012 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:41.565001 13708 raft_consensus.cc:3037] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:41.533641 13689 consensus_queue.cc:237] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
W20260809 15:52:41.569644 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 97423.3us
I20260809 15:52:41.588936 10978 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:41.539227 13723 consensus_queue.cc:1035] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.033s
I20260809 15:52:41.544508 13820 raft_consensus.cc:3037] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
W20260809 15:52:41.586865 10811 leader_election.cc:334] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.802s (SENT)
I20260809 15:52:41.571913 13334 raft_consensus.cc:2781] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:41.565322 13437 raft_consensus.cc:2781] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:41.594375 10811 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:41.581684 13522 raft_consensus.cc:2781] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:41.576054 13640 raft_consensus.cc:686] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:41.604592 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:41.593271 13334 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:41.581257 13464 consensus_queue.cc:1035] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260809 15:52:41.607442 10840 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:41.599736 13437 raft_consensus.cc:686] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:41.617857 11014 raft_consensus.cc:1261] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260809 15:52:41.630522 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:41.631181 11016 raft_consensus.cc:3037] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 5
I20260809 15:52:41.636540 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:41.637171 10841 raft_consensus.cc:2454] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:52:41.597853 13836 raft_consensus.cc:2781] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election won for term 5
W20260809 15:52:41.642625 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 74983.4us
I20260809 15:52:41.605418 13522 raft_consensus.cc:686] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:41.646507 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:41.646808 10839 raft_consensus.cc:1226] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260809 15:52:41.654534 10840 raft_consensus.cc:2396] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260809 15:52:41.662151 10843 raft_consensus.cc:2454] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
W20260809 15:52:41.631174 13826 log.cc:924] Time spent T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.066s user 0.000s sys 0.000s
I20260809 15:52:41.664676 11014 raft_consensus.cc:1261] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
W20260809 15:52:41.648334 13829 log.cc:924] Time spent T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f: Append to log took a long time: real 0.053s user 0.000s sys 0.000s
I20260809 15:52:41.591838 13806 consensus_queue.cc:260] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 3.2, Last appended by leader: 2, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:41.607816 13823 raft_consensus.cc:3037] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:41.625978 13334 raft_consensus.cc:3037] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:41.638417 13676 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:41.657552 13820 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.673615 13810 consensus_queue.cc:1035] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:52:41.675240 13806 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:41.667536 13574 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:41.679836 11017 raft_consensus.cc:1261] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 4 index: 2. (index mismatch)
I20260809 15:52:41.672168 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:41.693079 11014 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:41.643050 13473 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:41.687083 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:41.655064 13836 raft_consensus.cc:686] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 5 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:41.662580 13299 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:41.691408 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:41.698525 10843 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 6
I20260809 15:52:41.697736 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:41.699288 11640 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:41.696412 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:41.698448 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:41.682510 13650 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:41.688094 13656 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:41.705852 10838 raft_consensus.cc:2454] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:41.708473 13344 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:41.709388 11633 raft_consensus.cc:1261] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260809 15:52:41.700228 13473 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.703975 13679 consensus_queue.cc:1035] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.015s
I20260809 15:52:41.654155 13575 consensus_queue.cc:237] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:41.692759 13676 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.703280 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:41.716586 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:41.717834 10920 raft_consensus.cc:1261] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:41.664623 13830 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.035s
I20260809 15:52:41.720968 11016 raft_consensus.cc:2454] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:41.682787 13819 raft_consensus.cc:686] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:41.702791 13299 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.710433 13437 consensus_queue.cc:237] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.734530 11640 raft_consensus.cc:2454] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:41.756904 10809 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:41.738981 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 76829.2us
I20260809 15:52:41.740156 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:41.697865 13574 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:41.752839 11639 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:41.757562 10893 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:41.760804 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:41.765003 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:41.827459 10930 raft_consensus.cc:1226] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:41.716607 13522 consensus_queue.cc:237] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.829658 11640 raft_consensus.cc:2379] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 9: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:41.840464 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:41.845679 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:41.761621 13656 raft_consensus.cc:2781] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:41.849898 11641 raft_consensus.cc:1226] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:41.846328 11640 raft_consensus.cc:2454] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:41.828119 13503 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:41.851217 13464 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:41.852281 11642 raft_consensus.cc:3032] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:41.859356 11642 raft_consensus.cc:729] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:41.842522 13653 raft_consensus.cc:2781] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election won for term 4
W20260809 15:52:41.856887 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 97096.3us
I20260809 15:52:41.860249 11014 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:41.843914 13813 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:41.727805 13640 consensus_queue.cc:237] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:41.718691 13836 consensus_queue.cc:237] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:41.865427 10922 raft_consensus.cc:1226] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:41.851390 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:41.917904 13299 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:41.862043 13656 raft_consensus.cc:686] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:41.916082 10978 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:41.901059 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:41.869441 13503 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.769527 13837 consensus_queue.cc:1035] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.108s
I20260809 15:52:41.944689 11639 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:41.913230 13574 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:41.914984 13473 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:41.949008 11012 raft_consensus.cc:2454] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:41.864645 11642 consensus_queue.cc:260] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.988497 11642 raft_consensus.cc:3037] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:41.905381 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
W20260809 15:52:41.960420 13826 log.cc:924] Time spent T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.090s user 0.000s sys 0.000s
I20260809 15:52:41.859784 10921 raft_consensus.cc:2396] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260809 15:52:41.906390 13653 raft_consensus.cc:686] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:41.926961 10924 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:41.857787 13820 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:41.945482 13809 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:41.960680 13839 log.cc:924] Time spent T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.098s user 0.000s sys 0.000s
I20260809 15:52:41.986451 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:41.966845 13845 log.cc:924] Time spent T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.121s user 0.001s sys 0.000s
W20260809 15:52:41.989538 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 73581.7us
I20260809 15:52:41.991432 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:42.003149 11639 raft_consensus.cc:2354] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4.
I20260809 15:52:41.864234 13464 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.993209 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:41.995623 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:42.023643 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:42.012705 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:42.005730 13762 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.026841 13762 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.028927 13762 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:42.029971 10840 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 4
I20260809 15:52:42.030588 11640 raft_consensus.cc:2354] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4.
I20260809 15:52:41.960949 13708 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:41.964366 13813 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:41.993124 13574 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:41.988675 13629 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:41.981814 13334 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.034119 11639 raft_consensus.cc:2396] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260809 15:52:42.055106 10978 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:41.993284 13689 raft_consensus.cc:2781] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:52:42.057281 13676 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:42.060223 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
W20260809 15:52:42.053128 13594 consensus_meta.cc:287] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.537s user 0.010s sys 0.000s
I20260809 15:52:41.918346 13819 consensus_queue.cc:237] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:42.060868 13464 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:42.063944 10927 raft_consensus.cc:2379] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 4: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:41.999172 13810 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.067862 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:42.069428 11642 raft_consensus.cc:1261] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:42.072252 10840 raft_consensus.cc:2454] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:42.073982 11606 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
W20260809 15:52:42.062558 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 59775us
I20260809 15:52:42.052452 13503 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:42.050663 13828 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.080080 10926 raft_consensus.cc:2379] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 4: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:42.052235 11632 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:52:42.122732 10893 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:42.125133 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:42.126224 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 426772us
I20260809 15:52:42.126660 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:42.130257 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:42.131114 11013 raft_consensus.cc:2454] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:42.134574 11642 raft_consensus.cc:2396] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260809 15:52:42.064878 13689 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:42.143766 11018 raft_consensus.cc:1261] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260809 15:52:42.073477 13820 raft_consensus.cc:2781] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:42.149819 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:42.005398 13809 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.153592 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:42.140565 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:42.121675 13679 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.122838 13464 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.051065 13653 consensus_queue.cc:237] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.162984 11634 raft_consensus.cc:2379] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 3: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:42.059331 13629 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:42.188892 10893 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:42.124277 13838 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:42.148612 13344 raft_consensus.cc:2781] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:42.223989 11639 raft_consensus.cc:1226] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260809 15:52:42.186894 11640 raft_consensus.cc:2454] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:42.179322 10978 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:42.151144 13689 raft_consensus.cc:3037] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:42.073046 13594 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:42.198762 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 124853us
I20260809 15:52:42.067528 13574 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:42.200295 13823 consensus_meta.cc:287] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.507s user 0.008s sys 0.002s
I20260809 15:52:42.189131 11633 raft_consensus.cc:1226] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:42.199318 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:42.142441 13828 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:42.162911 13820 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:42.123395 13810 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.079035 13656 consensus_queue.cc:237] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:42.258222 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 78989.8us
I20260809 15:52:42.195498 13299 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.233048 13676 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.239688 11639 raft_consensus.cc:1261] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260809 15:52:42.263716 11639 pending_rounds.cc:76] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9: Aborting all ops after (but not including) 7
I20260809 15:52:42.263985 11639 pending_rounds.cc:97] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260809 15:52:42.264274 11639 raft_consensus.cc:2866] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:42.240097 11640 raft_consensus.cc:1261] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260809 15:52:42.266171 11640 pending_rounds.cc:76] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9: Aborting all ops after (but not including) 7
I20260809 15:52:42.245561 11018 raft_consensus.cc:2379] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 7: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:42.244560 11604 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:42.253118 11641 raft_consensus.cc:2454] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
W20260809 15:52:42.234936 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 46142.2us
I20260809 15:52:42.230383 11606 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:42.268004 11640 pending_rounds.cc:97] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260809 15:52:42.268595 11640 raft_consensus.cc:2866] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:42.198041 13679 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:42.242499 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:42.243850 13823 raft_consensus.cc:2726] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:42.244347 13344 raft_consensus.cc:686] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:42.255421 13820 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:42.223104 13464 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.319579 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:42.320227 10848 raft_consensus.cc:3037] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:42.317548 10838 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:42.274914 13830 raft_consensus.cc:2781] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:42.231277 13838 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:42.347338 10978 leader_election.cc:302] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:42.355105 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 124812us
I20260809 15:52:42.312937 13708 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:42.275027 13299 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.274567 13676 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:42.371701 10843 consensus_meta.cc:287] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.665s user 0.010s sys 0.001s
I20260809 15:52:42.359978 13813 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:42.358796 13334 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:42.369016 13825 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.474284 13594 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:42.473942 13825 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.363843 13823 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.475695 13823 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.478296 13855 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.478929 13855 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.480278 13679 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:42.480825 13679 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.481314 13679 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.481590 13857 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:42.482512 13464 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:42.482323 13857 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.483064 13464 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.483497 13464 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.477681 13825 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:42.400806 10848 raft_consensus.cc:2454] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:42.366219 13858 raft_consensus.cc:2781] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:52:42.404742 10843 raft_consensus.cc:2454] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
W20260809 15:52:42.403604 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 164263us
I20260809 15:52:42.369035 13503 raft_consensus.cc:2781] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:42.369663 13860 raft_consensus.cc:2726] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:42.464511 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:42.470731 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:42.470990 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:42.471827 13848 log.cc:924] Time spent T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.273s user 0.001s sys 0.000s
W20260809 15:52:42.472883 13843 log.cc:924] Time spent T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.568s user 0.004s sys 0.000s
I20260809 15:52:42.475160 13629 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:42.492511 11604 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:42.506884 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 150371us
I20260809 15:52:42.509519 11018 raft_consensus.cc:2396] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 5 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 15:52:42.513849 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:42.507535 10844 raft_consensus.cc:1261] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260809 15:52:42.522122 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 199940us
I20260809 15:52:42.524377 10978 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:42.522380 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:42.477725 13823 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:42.530769 11016 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 6
W20260809 15:52:42.511600 13851 log.cc:924] Time spent T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.275s user 0.001s sys 0.000s
I20260809 15:52:42.538059 11013 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 1.
I20260809 15:52:42.539794 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:42.540397 11013 raft_consensus.cc:2454] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:42.541198 11606 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:42.530397 13858 raft_consensus.cc:686] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:42.544134 13858 consensus_queue.cc:237] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.546187 13861 raft_consensus.cc:2726] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:42.546885 13868 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.547394 13868 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:42.548727 11642 raft_consensus.cc:1226] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:42.549440 13868 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:42.551013 13856 log.cc:924] Time spent T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.227s user 0.000s sys 0.000s
W20260809 15:52:42.553130 13806 consensus_meta.cc:287] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.856s user 0.008s sys 0.000s
I20260809 15:52:42.553911 10841 raft_consensus.cc:1261] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 4 index: 2. (term mismatch)
I20260809 15:52:42.554255 10841 pending_rounds.cc:76] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536: Aborting all ops after (but not including) 1
I20260809 15:52:42.554315 13869 raft_consensus.cc:2781] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:42.554567 10841 pending_rounds.cc:97] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536: Aborting uncommitted NO_OP operation due to leader change: 3.2
I20260809 15:52:42.554796 13869 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:42.555251 13869 raft_consensus.cc:3037] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:42.554852 10841 raft_consensus.cc:2866] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:42.557238 11018 raft_consensus.cc:1226] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:42.515791 13825 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.572125 13679 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:42.573020 13838 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:42.574028 13464 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:42.587911 13859 consensus_queue.cc:1046] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260809 15:52:42.576812 13857 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:42.574769 13828 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:42.589303 13859 raft_consensus.cc:3032] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:42.589648 13825 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.582644 13868 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.590970 13868 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.591936 13825 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:42.575734 13855 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:42.576278 13809 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:42.522758 13503 raft_consensus.cc:686] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:42.593938 13503 consensus_queue.cc:237] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.594638 13868 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:42.588258 13628 raft_consensus.cc:2726] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:42.455957 13344 consensus_queue.cc:237] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.507334 13864 raft_consensus.cc:2781] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:52:42.584102 13861 raft_consensus.cc:2781] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:42.599244 11606 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
W20260809 15:52:42.599630 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 75364.9us
I20260809 15:52:42.600484 10978 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:42.581722 11016 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
W20260809 15:52:42.600664 10811 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.594s user 0.001s sys 0.007s
W20260809 15:52:42.616838 10811 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.610s user 0.002s sys 0.007s
I20260809 15:52:42.589807 13859 raft_consensus.cc:729] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
W20260809 15:52:42.611568 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 295861us
I20260809 15:52:42.584579 13810 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:42.597301 13574 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:42.680754 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:42.684520 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:42.611157 13864 raft_consensus.cc:482] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:52:42.685432 10891 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.539s user 0.004s sys 0.013s
W20260809 15:52:42.685899 10891 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.539s user 0.004s sys 0.013s
I20260809 15:52:42.686955 10893 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
W20260809 15:52:42.688757 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 89610.5us
I20260809 15:52:42.690936 11606 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:42.686702 11016 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:42.693207 11606 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:42.694949 10809 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:42.696329 10809 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:42.697422 10809 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:42.701987 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 101603us
I20260809 15:52:42.704571 11018 raft_consensus.cc:2396] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260809 15:52:42.671805 13679 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.708212 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:42.718829 11017 raft_consensus.cc:2454] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:42.713629 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:42.679301 13861 raft_consensus.cc:686] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:42.683228 13628 raft_consensus.cc:2726] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:42.730581 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:42.729305 11016 raft_consensus.cc:1261] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 15:52:42.731233 10841 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 9.
I20260809 15:52:42.735383 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:42.736193 10843 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:42.704807 13857 raft_consensus.cc:2781] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election won for term 4
W20260809 15:52:42.745702 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 55086.2us
I20260809 15:52:42.688356 13864 raft_consensus.cc:3037] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:42.708662 13828 raft_consensus.cc:2781] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:42.704027 13809 raft_consensus.cc:2781] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:42.748597 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:42.748871 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:42.747756 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:42.813186 11636 raft_consensus.cc:3037] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:42.751021 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:42.751210 10891 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:42.707584 13676 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:42.818459 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:42.724080 13762 raft_consensus.cc:2726] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260809 15:52:42.821378 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 105037us
W20260809 15:52:42.806440 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 88494.3us
I20260809 15:52:42.812177 10848 raft_consensus.cc:2454] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:42.812767 10838 raft_consensus.cc:2454] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:42.814239 11641 raft_consensus.cc:2454] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:42.733392 13299 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:42.751863 10841 raft_consensus.cc:1261] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260809 15:52:42.826520 10841 pending_rounds.cc:76] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536: Aborting all ops after (but not including) 5
I20260809 15:52:42.826841 10841 pending_rounds.cc:97] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536: Aborting uncommitted NO_OP operation due to leader change: 4.6
W20260809 15:52:42.826992 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 77421.6us
I20260809 15:52:42.822533 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:42.745635 13574 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.832733 13832 raft_consensus.cc:2781] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:42.833171 13832 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:42.833107 13574 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:42.834649 11606 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:42.835484 13574 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:42.822443 10810 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.571s user 0.003s sys 0.002s
W20260809 15:52:42.836136 10810 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.850s user 0.003s sys 0.003s
I20260809 15:52:42.836510 13893 raft_consensus.cc:2781] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:42.836973 13893 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:42.837390 13893 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:42.838339 13883 raft_consensus.cc:2781] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:42.838785 13883 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:42.839128 13883 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:42.839414 13594 consensus_queue.cc:1046] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:42.840314 13594 raft_consensus.cc:3032] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:42.840636 10844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 697) took 1089 ms. Trace:
W20260809 15:52:42.840912 13844 log.cc:924] Time spent T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.740s user 0.005s sys 0.009s
I20260809 15:52:42.841087 10844 rpcz_store.cc:276] 0809 15:52:41.751347 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:41.751467 (+ 120us) service_pool.cc:227] Handling call
0809 15:52:42.493048 (+741581us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:42.524021 (+ 30973us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:42.840385 (+316364us) spinlock_profiling.cc:243] Waited 732 ms on lock 0x7b54001f13d0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:42.840553 (+ 168us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1684188032}
I20260809 15:52:42.827113 10841 raft_consensus.cc:2866] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260809 15:52:42.842207 13865 log.cc:924] Time spent T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.190s user 0.000s sys 0.001s
I20260809 15:52:42.842391 11637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 841) took 1156 ms. Trace:
I20260809 15:52:42.843578 13822 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.842821 11637 rpcz_store.cc:276] 0809 15:52:41.685769 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:41.685896 (+ 127us) service_pool.cc:227] Handling call
0809 15:52:41.686246 (+ 350us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:41.700281 (+ 14035us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:41.700311 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:41.700733 (+ 422us) log.cc:842] Serialized 26 byte log entry
0809 15:52:41.833011 (+132278us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:41.833057 (+ 46us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:41.833131 (+ 74us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:42.840161 (+1007030us) spinlock_profiling.cc:243] Waited 2.9 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:52:42.842059 (+ 1898us) raft_consensus.cc:1671] finished
0809 15:52:42.842096 (+ 37us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:42.842305 (+ 209us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8140,"raft.queue_time_us":7460,"raft.run_cpu_time_us":135,"raft.run_wall_time_us":4831,"spinlock_wait_cycles":6673664,"thread_start_us":115929,"threads_started":1,"wal-append.queue_time_us":293255}
I20260809 15:52:42.844118 13822 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:42.844738 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:42.846267 13873 raft_consensus.cc:3037] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:42.847040 13872 consensus_queue.cc:1035] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.249s
W20260809 15:52:42.848587 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 97485.1us
I20260809 15:52:42.849053 13863 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.849340 10891 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:42.849838 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:42.850412 13871 raft_consensus.cc:2726] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:42.850689 10840 raft_consensus.cc:2396] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260809 15:52:42.850931 10891 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:42.725123 13629 consensus_queue.cc:1035] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.094s
I20260809 15:52:42.851090 13871 raft_consensus.cc:2726] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:42.852108 13629 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
W20260809 15:52:42.830945 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 115894us
I20260809 15:52:42.858932 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:42.832247 13894 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.832378 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:42.863180 11013 raft_consensus.cc:2379] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 5: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:42.831082 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:42.830859 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:42.830855 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:42.830268 11640 raft_consensus.cc:2379] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 5: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:42.856545 11636 raft_consensus.cc:2454] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:42.833712 13832 raft_consensus.cc:3037] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:42.864439 11641 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 9.
I20260809 15:52:42.871026 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
W20260809 15:52:42.808310 13837 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.578s user 0.017s sys 0.012s
I20260809 15:52:42.877000 10893 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:42.802199 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:42.883008 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:42.868953 10927 raft_consensus.cc:2396] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 10 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260809 15:52:42.875537 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:42.876157 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:42.884011 11017 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 8.
I20260809 15:52:42.819746 13809 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:42.730923 13679 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:42.871779 10842 raft_consensus.cc:2454] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:42.868031 10978 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:42.849567 13863 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:42.882634 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:42.875452 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:42.885738 10844 raft_consensus.cc:1261] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:42.882566 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:42.884215 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:42.856669 13823 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.888526 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:42.802208 13650 consensus_queue.cc:1035] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.123s
I20260809 15:52:42.888860 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:42.888890 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:42.886713 10843 raft_consensus.cc:2454] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:42.823987 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:42.893524 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:42.928582 10840 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:42.826442 13857 raft_consensus.cc:686] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:42.893556 10978 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:42.895157 11636 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 5 LEARNER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:42.897884 13823 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.888267 13809 raft_consensus.cc:3037] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:42.898355 13588 raft_consensus.cc:2781] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260809 15:52:42.938766 13879 log.cc:924] Time spent T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.115s user 0.000s sys 0.000s
I20260809 15:52:42.939018 13588 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:42.939244 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:42.939432 13588 raft_consensus.cc:3037] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:42.939857 11636 raft_consensus.cc:3037] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:42.941047 13810 consensus_queue.cc:1046] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 15:52:42.942219 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:42.942306 13878 consensus_queue.cc:1046] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.138s
I20260809 15:52:42.943321 13878 raft_consensus.cc:3032] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:52:42.943770 13878 raft_consensus.cc:729] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
W20260809 15:52:42.944000 13875 log.cc:924] Time spent T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.541s user 0.006s sys 0.007s
I20260809 15:52:42.840672 13594 raft_consensus.cc:729] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
W20260809 15:52:42.945426 13866 log.cc:924] Time spent T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.541s user 0.000s sys 0.008s
I20260809 15:52:42.945956 13594 consensus_queue.cc:260] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.869493 13473 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.947300 13473 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.948719 13874 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.949355 13473 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:42.949194 13874 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:42.923837 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:42.883188 13837 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.653s user 0.017s sys 0.012s
I20260809 15:52:42.906199 11013 raft_consensus.cc:2379] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 5: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:42.924710 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:42.955500 10838 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:42.932070 10978 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:42.898290 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:42.935992 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:42.888033 13838 raft_consensus.cc:2741] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260809 15:52:42.937022 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:42.958765 10840 raft_consensus.cc:2396] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260809 15:52:42.901087 13575 consensus_queue.cc:1035] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260809 15:52:42.777189 13859 consensus_queue.cc:260] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.938467 13890 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.927433 13861 consensus_queue.cc:237] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.942619 11633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 643) took 1179 ms. Trace:
I20260809 15:52:42.894928 13628 raft_consensus.cc:482] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:42.951774 11017 raft_consensus.cc:2454] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:42.923348 11635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 651) took 1091 ms. Trace:
I20260809 15:52:42.952113 11640 raft_consensus.cc:2379] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 4: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:42.885866 11010 raft_consensus.cc:2454] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:52:42.978371 11604 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:42.971824 11633 rpcz_store.cc:276] 0809 15:52:41.763608 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:41.769359 (+ 5751us) service_pool.cc:227] Handling call
0809 15:52:42.146018 (+376659us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:42.244883 (+ 98865us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:42.942406 (+697523us) spinlock_profiling.cc:243] Waited 292 ms on lock 0x7b54006707d0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:42.942550 (+ 144us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":671608192}
I20260809 15:52:42.980504 13628 raft_consensus.cc:504] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.982216 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:42.982556 13830 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:42.971822 13640 raft_consensus.cc:2741] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260809 15:52:42.982890 11011 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
I20260809 15:52:42.882786 13813 consensus_queue.cc:1035] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.060s
I20260809 15:52:42.900277 11632 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:42.929518 13828 raft_consensus.cc:686] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:42.956171 13874 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:42.967617 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:42.900529 13822 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:42.855079 13798 raft_consensus.cc:2932] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Committing config change with OpId 3.5: config changed from index 4 to 5, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:42.982676 13628 leader_election.cc:288] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:42.952898 11014 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:42.986294 10978 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:42.989827 11636 raft_consensus.cc:2454] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:42.977334 11635 rpcz_store.cc:276] 0809 15:52:41.831445 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:42.197622 (+366177us) service_pool.cc:227] Handling call
0809 15:52:42.233708 (+ 36086us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:42.233996 (+ 288us) raft_consensus.cc:1520] Early marking committed up to index 6
0809 15:52:42.259250 (+ 25254us) log.cc:842] Serialized 14 byte log entry
0809 15:52:42.849280 (+590030us) spinlock_profiling.cc:243] Waited 1.28 ms on lock 0x7b18003c34c0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f79665ad345 00007f796f3e4a39 00007f79665ca8bb 00007f79665c8cbf 00007f79665cae22 00007f79691745bb 00007f796917a063 00007f796917a57b 00007f79694fd88a
0809 15:52:42.922991 (+ 73711us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:42.923052 (+ 61us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:42.923109 (+ 57us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:42.923183 (+ 74us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:42.923291 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":25212,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":34,"raft.queue_time_us":27073,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":8746,"spinlock_wait_cycles":2946944,"thread_start_us":46646,"threads_started":1}
I20260809 15:52:42.990181 13830 raft_consensus.cc:3037] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:42.968816 13836 raft_consensus.cc:2781] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:42.896337 13503 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.058s
I20260809 15:52:43.001864 11018 raft_consensus.cc:2396] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 7 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260809 15:52:43.006876 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:43.015781 11011 raft_consensus.cc:2454] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:43.016933 11604 leader_election.cc:302] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:43.012944 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 134993us
I20260809 15:52:42.889575 13894 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:42.984552 13838 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:42.994036 13473 raft_consensus.cc:2781] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:43.012106 11641 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
W20260809 15:52:43.024757 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 163132us
I20260809 15:52:43.026503 11606 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:43.019085 13820 consensus_meta.cc:287] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.672s user 0.009s sys 0.005s
I20260809 15:52:43.016439 13874 raft_consensus.cc:2741] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260809 15:52:43.033531 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 47360.4us
I20260809 15:52:43.035722 10809 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:43.022464 13855 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:43.025323 13473 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:42.975147 13890 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:43.024420 13836 raft_consensus.cc:686] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:42.990811 13857 consensus_queue.cc:237] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.050180 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:43.050329 13828 consensus_queue.cc:237] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:43.051167 13876 log.cc:924] Time spent T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.067s user 0.000s sys 0.000s
W20260809 15:52:43.051832 13877 log.cc:924] Time spent T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.118s user 0.000s sys 0.001s
I20260809 15:52:43.052460 13435 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:43.042835 13900 raft_consensus.cc:2781] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:43.027990 13838 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:43.071938 10811 leader_election.cc:334] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.633s (SENT)
I20260809 15:52:43.047827 13473 raft_consensus.cc:3037] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:43.089843 13903 raft_consensus.cc:2781] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:43.090250 13853 raft_consensus.cc:3032] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:52:43.091168 13899 consensus_queue.cc:1046] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260809 15:52:43.093559 13907 consensus_queue.cc:1035] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:43.004812 13878 consensus_queue.cc:260] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.070940 13863 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:43.101078 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 52193.2us
I20260809 15:52:43.034554 13820 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.072504 13629 raft_consensus.cc:2726] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:43.100262 13679 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:43.159614 10978 leader_election.cc:398] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Vote denied by peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783) with higher term. Message: Invalid argument: T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5.
I20260809 15:52:43.096757 11637 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:43.191294 10978 leader_election.cc:401] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260809 15:52:43.077636 13900 raft_consensus.cc:482] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:43.042908 13855 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.108592 13853 raft_consensus.cc:729] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:43.188943 10893 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:43.097900 13903 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 15:52:43.172175 13845 log.cc:924] Time spent T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.077s user 0.000s sys 0.000s
I20260809 15:52:43.201824 10893 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
W20260809 15:52:43.176962 13869 consensus_meta.cc:287] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.621s user 0.004s sys 0.008s
I20260809 15:52:43.201876 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:43.206197 11639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 852) took 1018 ms. Trace:
I20260809 15:52:43.210126 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:43.210261 10893 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:43.211274 13650 raft_consensus.cc:2781] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:43.211386 10893 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:43.211692 13650 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:43.212054 13650 raft_consensus.cc:3037] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:43.212292 13867 raft_consensus.cc:2781] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:43.212726 13867 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:43.213191 13867 raft_consensus.cc:3037] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
W20260809 15:52:43.214216 13896 log.cc:924] Time spent T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.073s user 0.000s sys 0.001s
I20260809 15:52:43.214287 13858 raft_consensus.cc:2781] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:43.214754 13858 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:43.215123 13858 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
W20260809 15:52:43.215821 13898 log.cc:924] Time spent T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.080s user 0.001s sys 0.000s
I20260809 15:52:43.216934 13819 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
W20260809 15:52:43.217841 13887 log.cc:924] Time spent T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.142s user 0.000s sys 0.001s
I20260809 15:52:43.217494 13819 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:43.219735 13819 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:43.222661 13894 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:43.198697 13861 raft_consensus.cc:2726] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783) with higher term. Message: Invalid argument: T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5.
I20260809 15:52:43.224084 13836 consensus_queue.cc:237] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.225558 13858 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:43.226564 13629 consensus_queue.cc:1035] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260809 15:52:43.227999 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 163296us
W20260809 15:52:43.228820 10811 leader_election.cc:334] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.672s (SENT)
I20260809 15:52:43.229290 10811 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:43.230916 13899 raft_consensus.cc:2726] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:43.176429 13628 raft_consensus.cc:2726] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 15:52:43.094713 13435 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.212761 10893 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9
W20260809 15:52:43.240622 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 81201.4us
I20260809 15:52:43.241778 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:43.206665 11639 rpcz_store.cc:276] 0809 15:52:42.187911 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:42.265194 (+ 77283us) service_pool.cc:227] Handling call
0809 15:52:42.265526 (+ 332us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:42.400583 (+135057us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:42.400639 (+ 56us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:42.401098 (+ 459us) log.cc:842] Serialized 26 byte log entry
0809 15:52:42.655438 (+254340us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:42.655478 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:42.655545 (+ 67us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:43.205989 (+550444us) raft_consensus.cc:1671] finished
0809 15:52:43.206009 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:43.206141 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8070,"raft.queue_time_us":47418,"raft.run_cpu_time_us":182,"raft.run_wall_time_us":31117,"thread_start_us":376674,"threads_started":2,"wal-append.queue_time_us":481117}
I20260809 15:52:43.242525 10893 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:43.242578 10838 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
W20260809 15:52:43.246842 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 220419us
I20260809 15:52:43.249536 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:43.250209 10843 raft_consensus.cc:2454] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
W20260809 15:52:43.250880 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 151769us
I20260809 15:52:43.251972 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:43.252645 10842 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:43.252763 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:43.253288 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:43.253532 11015 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:43.254675 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:43.254851 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:43.255000 11604 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:43.255389 11633 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 2.
I20260809 15:52:43.256347 11606 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:43.256549 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:43.257369 11633 raft_consensus.cc:2454] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:43.258019 11012 raft_consensus.cc:1226] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:43.263407 13435 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:43.271261 13629 raft_consensus.cc:2781] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:43.271708 13629 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:43.236019 13869 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.273711 13869 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:43.277149 13838 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:43.200099 13900 raft_consensus.cc:3037] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
W20260809 15:52:43.263370 13689 consensus_meta.cc:287] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 1.024s user 0.013s sys 0.000s
I20260809 15:52:43.267043 10978 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:43.258185 13825 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:43.293254 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:43.266422 13464 raft_consensus.cc:2781] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:43.261027 11639 raft_consensus.cc:2379] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 9: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:43.272074 13629 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:43.263290 13894 raft_consensus.cc:2781] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:43.224038 13650 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.262821 13906 raft_consensus.cc:2781] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:43.280267 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:43.328428 10840 raft_consensus.cc:3037] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:43.281309 11606 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:43.235059 13679 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 15:52:43.259985 13878 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:43.202178 13903 raft_consensus.cc:3037] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:43.269043 10811 leader_election.cc:398] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Vote denied by peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685) with higher term. Message: Invalid argument: T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4.
W20260809 15:52:43.337960 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 70996.1us
I20260809 15:52:43.269129 10893 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:43.314602 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:43.326553 13850 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:43.327338 13858 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:43.336490 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:43.338157 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 69301us
I20260809 15:52:43.292877 13435 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:43.339095 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 70201.2us
I20260809 15:52:43.296959 11641 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:43.339412 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:43.295495 13464 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 15:52:43.340721 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 59517us
I20260809 15:52:43.341717 10927 raft_consensus.cc:2454] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:43.341980 10811 leader_election.cc:398] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Vote denied by peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685) with higher term. Message: Invalid argument: T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4.
I20260809 15:52:43.338812 10978 leader_election.cc:302] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:43.295790 13838 raft_consensus.cc:2726] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:43.351456 11015 raft_consensus.cc:3037] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:43.353094 10811 leader_election.cc:401] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Cancelling election due to peer responding with higher term
I20260809 15:52:43.353914 10840 raft_consensus.cc:2454] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:43.328260 13894 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:43.332938 13906 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:43.361034 11606 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:43.334774 13679 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:43.363539 10893 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:43.364768 11641 raft_consensus.cc:1261] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 3 index: 6. (index mismatch)
I20260809 15:52:43.343250 13522 raft_consensus.cc:2781] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:43.295388 13689 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.354949 13464 raft_consensus.cc:3037] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:43.298848 13825 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:43.363078 13894 raft_consensus.cc:3037] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:43.356298 13435 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:43.357133 13861 raft_consensus.cc:2726] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:43.377557 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:43.365561 13906 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:43.387365 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:43.387984 10848 raft_consensus.cc:3037] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:43.384943 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:43.368021 13679 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Advancing to term 10
I20260809 15:52:43.387055 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:43.362267 13828 raft_consensus.cc:2726] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 3. Reason: Vote denied by peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685) with higher term. Message: Invalid argument: T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4.
I20260809 15:52:43.393412 11016 raft_consensus.cc:1226] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:43.310966 13864 raft_consensus.cc:504] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.313582 13883 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.373566 13522 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:43.395836 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:43.398494 10930 raft_consensus.cc:2454] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:43.356771 13907 consensus_queue.cc:1035] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260809 15:52:43.408308 10848 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:43.416657 10978 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:43.406669 13882 raft_consensus.cc:2781] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:43.401054 13522 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:43.288951 13853 consensus_queue.cc:260] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:43.407680 13893 consensus_meta.cc:287] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.565s user 0.004s sys 0.006s
I20260809 15:52:43.442013 13823 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:43.455603 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:43.458926 11011 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:43.459192 13893 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.423481 13650 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:43.461159 13893 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:43.461748 11635 raft_consensus.cc:1226] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:43.450503 13628 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:43.462883 13628 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.449105 13825 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:43.464110 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:43.448546 13594 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:43.428951 13864 leader_election.cc:288] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:43.424275 13882 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:43.466583 13882 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:43.455199 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:43.434186 13920 raft_consensus.cc:2781] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:43.470402 13920 raft_consensus.cc:686] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:43.470551 11633 raft_consensus.cc:1261] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260809 15:52:43.423092 13861 raft_consensus.cc:2741] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won
I20260809 15:52:43.342733 13850 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.471298 13920 consensus_queue.cc:237] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.472903 13850 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:43.398916 13820 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:43.473603 13855 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:43.434422 13913 raft_consensus.cc:2726] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:52:43.443506 13883 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:43.475253 11640 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:52:43.476011 11011 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:43.439551 13809 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.441207 13435 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:43.459628 13832 consensus_meta.cc:287] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.570s user 0.009s sys 0.000s
W20260809 15:52:43.480749 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 56362us
I20260809 15:52:43.470204 13906 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.477054 10809 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:43.440114 13629 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:43.484825 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 82820.7us
I20260809 15:52:43.433904 11015 raft_consensus.cc:2454] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:43.426730 10978 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:43.454780 11642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 649) took 1623 ms. Trace:
I20260809 15:52:43.443835 10841 raft_consensus.cc:2454] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:43.498346 11642 rpcz_store.cc:276] 0809 15:52:41.830905 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:42.137094 (+306189us) service_pool.cc:227] Handling call
0809 15:52:42.548556 (+411462us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:42.549140 (+ 584us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:43.454599 (+905459us) spinlock_profiling.cc:243] Waited 411 ms on lock 0x7b54005221d0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:43.454723 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":945563520}
W20260809 15:52:43.455521 13588 consensus_meta.cc:287] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.516s user 0.008s sys 0.001s
I20260809 15:52:43.509738 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:43.508687 11636 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8"
dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34950
I20260809 15:52:43.513296 11636 raft_consensus.cc:595] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:43.513677 11636 raft_consensus.cc:603] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 15:52:43.502200 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:43.504653 10840 raft_consensus.cc:1261] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:43.518600 10847 raft_consensus.cc:2379] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 3: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
W20260809 15:52:43.544188 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 62550.2us
I20260809 15:52:43.544909 10811 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:43.545377 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:43.447322 13859 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:43.546193 10839 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
W20260809 15:52:43.556721 13928 log.cc:924] Time spent T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.132s user 0.000s sys 0.000s
I20260809 15:52:43.508668 13810 raft_consensus.cc:2781] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:43.572221 13810 raft_consensus.cc:686] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:43.573140 13810 consensus_queue.cc:237] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.555904 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:43.574391 11642 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:52:43.565685 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 2
I20260809 15:52:43.512248 13820 raft_consensus.cc:2741] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
I20260809 15:52:43.451711 13867 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:43.577526 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:52:43.569325 13522 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:43.571246 13903 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:43.566519 13629 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:43.518039 13869 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:43.591676 10893 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
W20260809 15:52:43.580158 10737 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:59504 (request call id 266) took 11363 ms (client timeout 15000 ms). Trace:
I20260809 15:52:43.580704 11606 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:43.549033 13907 raft_consensus.cc:2726] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:43.455322 13830 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.545495 13762 consensus_queue.cc:1035] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.039s
I20260809 15:52:43.594885 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
W20260809 15:52:43.595542 13922 log.cc:924] Time spent T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.194s user 0.001s sys 0.000s
I20260809 15:52:43.444383 13679 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.519520 13588 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.565004 13880 consensus_queue.cc:1035] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
W20260809 15:52:43.600283 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 173648us
I20260809 15:52:43.580518 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:43.631198 10843 raft_consensus.cc:1261] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260809 15:52:43.630668 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:43.640829 10843 pending_rounds.cc:76] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536: Aborting all ops after (but not including) 5
I20260809 15:52:43.625579 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:43.609150 13689 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:43.610348 13809 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:43.611778 11012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2283) took 1498 ms. Trace:
W20260809 15:52:43.588989 13798 consensus_meta.cc:287] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.513s user 0.010s sys 0.000s
I20260809 15:52:43.558044 13823 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260809 15:52:43.501282 13832 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.583540 13837 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.046s
I20260809 15:52:43.623710 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:43.746924 10841 raft_consensus.cc:3037] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
W20260809 15:52:43.649179 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 57596.2us
I20260809 15:52:43.742190 11012 rpcz_store.cc:276] 0809 15:52:42.113752 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:42.113874 (+ 122us) service_pool.cc:227] Handling call
0809 15:52:42.114225 (+ 351us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:43.267861 (+1153636us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:43.611537 (+343676us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b54001ddb54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:43.611703 (+ 166us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2576269184}
I20260809 15:52:43.599943 10811 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:43.596071 13850 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:43.621554 13924 log.cc:924] Time spent T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.214s user 0.000s sys 0.002s
I20260809 15:52:43.642144 11014 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 5
I20260809 15:52:43.610001 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:43.621970 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:43.641960 10840 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
W20260809 15:52:43.736342 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 95779.9us
I20260809 15:52:43.622546 10837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2186) took 1544 ms. Trace:
I20260809 15:52:43.641566 10843 pending_rounds.cc:97] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260809 15:52:43.753818 10978 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:43.755055 10978 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:43.753535 10837 rpcz_store.cc:276] 0809 15:52:42.077886 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:42.078063 (+ 177us) service_pool.cc:227] Handling call
0809 15:52:42.504297 (+426234us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:42.512339 (+ 8042us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:42.512378 (+ 39us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:42.512839 (+ 461us) log.cc:842] Serialized 26 byte log entry
0809 15:52:42.693427 (+180588us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:42.693464 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:42.693531 (+ 67us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:43.218980 (+525449us) raft_consensus.cc:1671] finished
0809 15:52:43.219001 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:43.622315 (+403314us) spinlock_profiling.cc:243] Waited 426 ms on lock 0x7b5400171bd0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:43.622491 (+ 176us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":7586,"raft.queue_time_us":8371,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":16441,"spinlock_wait_cycles":979199744,"thread_start_us":141102,"threads_started":1,"wal-append.queue_time_us":473737}
I20260809 15:52:43.630631 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:43.755828 13689 raft_consensus.cc:2726] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:43.756484 13689 raft_consensus.cc:2726] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:43.756640 10978 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:43.757413 13819 raft_consensus.cc:2726] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:43.754026 10843 raft_consensus.cc:2866] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:43.755923 11015 raft_consensus.cc:3037] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
W20260809 15:52:43.748314 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 154014us
I20260809 15:52:43.759949 10811 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:43.758563 10978 leader_election.cc:302] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: c7aa762318cb4e97875333f10679a9f9
W20260809 15:52:43.618407 13873 consensus_meta.cc:287] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.772s user 0.011s sys 0.000s
I20260809 15:52:43.616611 13906 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:43.752385 11633 raft_consensus.cc:3037] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:43.622525 13473 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.607125 13503 raft_consensus.cc:2781] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:43.742873 13893 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:43.614001 13464 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.762324 13936 raft_consensus.cc:2781] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:52:43.776643 13936 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:43.776988 13936 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:43.777698 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:43.759068 13878 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:43.778523 13878 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.778996 10921 raft_consensus.cc:1226] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260809 15:52:43.751770 13594 raft_consensus.cc:2741] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
I20260809 15:52:43.780485 13878 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:43.601825 13869 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.783700 13869 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:43.784479 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:43.785688 13832 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:43.786260 11010 raft_consensus.cc:1226] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:43.617370 13900 raft_consensus.cc:504] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.760253 13850 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:43.789083 11012 raft_consensus.cc:1261] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 15:52:43.789475 13900 leader_election.cc:288] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:43.789781 13850 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:43.791296 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:43.791631 10978 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:43.791883 11011 raft_consensus.cc:3037] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 7
I20260809 15:52:43.792443 13917 raft_consensus.cc:2726] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 15:52:43.794775 11606 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:43.795450 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:43.619885 13894 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.796588 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:43.796954 10841 raft_consensus.cc:2454] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:43.797260 13894 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:43.797683 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:43.620936 13830 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:43.798110 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:43.798704 11606 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
W20260809 15:52:43.800148 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 52540.6us
I20260809 15:52:43.800235 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
W20260809 15:52:43.800812 13910 log.cc:924] Time spent T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.623s user 0.001s sys 0.000s
I20260809 15:52:43.800873 10922 raft_consensus.cc:3037] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:43.765856 11632 raft_consensus.cc:2379] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 3: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:43.767684 13806 consensus_queue.cc:1035] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
I20260809 15:52:43.803447 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:43.768075 10811 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:43.804809 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:43.780545 10734 catalog_manager.cc:5495] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 2 to 4, leader changed from c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 4 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:43.808300 10841 raft_consensus.cc:2354] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7.
I20260809 15:52:43.619447 13867 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:43.822042 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:43.801107 10893 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:43.801079 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:43.808138 11606 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:43.828249 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:43.832341 11016 raft_consensus.cc:1261] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:43.829586 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:43.832752 13900 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:43.771132 13503 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:43.833563 13503 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:43.833315 13900 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.835268 13880 raft_consensus.cc:2726] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 15:52:43.837654 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:43.827306 10927 raft_consensus.cc:2454] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:43.838575 11012 raft_consensus.cc:2379] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 4: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:43.839598 10837 raft_consensus.cc:2379] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 5: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:43.826787 11011 raft_consensus.cc:2454] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
I20260809 15:52:43.837321 13853 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:43.854110 10809 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:43.854118 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:43.813371 11015 raft_consensus.cc:2454] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:43.819160 10734 catalog_manager.cc:5495] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 1 to 3, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 3 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
W20260809 15:52:43.856587 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 55536us
I20260809 15:52:43.770149 13809 raft_consensus.cc:2781] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:43.773070 13464 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:43.850201 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:43.771647 13679 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:43.765503 13473 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:43.629529 13882 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:43.858244 10734 catalog_manager.cc:5495] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
W20260809 15:52:43.859376 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 51296.6us
I20260809 15:52:43.772265 11633 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:43.811328 11014 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:43.866468 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:43.773969 13903 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:43.869940 10978 leader_election.cc:398] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Vote denied by peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) with higher term. Message: Invalid argument: T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7.
I20260809 15:52:43.891687 10978 leader_election.cc:401] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Cancelling election due to peer responding with higher term
I20260809 15:52:43.892156 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:43.770975 13522 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:43.771157 13588 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:43.849638 11013 raft_consensus.cc:1261] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
I20260809 15:52:43.813267 10840 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:43.874408 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:52:43.874823 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 112450us
I20260809 15:52:43.905356 10842 raft_consensus.cc:3037] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:43.905864 10811 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:43.805789 13873 raft_consensus.cc:2726] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:43.872824 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:43.791664 13832 raft_consensus.cc:2781] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:43.878854 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:43.907634 11015 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:43.883284 11016 raft_consensus.cc:2932] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:43.907989 11606 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:43.828243 13894 raft_consensus.cc:2726] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:43.909241 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:43.909508 10809 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:43.909898 10930 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:43.909852 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:43.884202 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:43.910799 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:43.911432 11013 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Advancing to term 10
I20260809 15:52:43.885315 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:43.886448 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:43.910825 10840 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:43.912308 10844 raft_consensus.cc:2379] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 5: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:43.881356 10841 raft_consensus.cc:1261] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260809 15:52:43.912846 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:43.879483 11604 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:43.883494 10734 catalog_manager.cc:5495] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 1 to 3, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 3 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:43.890892 10837 raft_consensus.cc:2354] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 2. Current term is 3.
I20260809 15:52:43.861934 10893 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:43.894701 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:43.872071 13867 raft_consensus.cc:2781] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:43.771450 13893 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.876559 13809 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:43.942353 13809 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:43.942623 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:43.943188 13867 raft_consensus.cc:686] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:43.944199 13867 consensus_queue.cc:237] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:43.944875 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:43.902772 13464 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:43.946615 13464 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:43.947956 13628 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:43.948156 13832 raft_consensus.cc:686] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:43.948478 13464 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:43.949173 13832 consensus_queue.cc:237] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.943224 10929 raft_consensus.cc:2454] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:43.902875 13689 raft_consensus.cc:2726] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: Vote denied by peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) with higher term. Message: Invalid argument: T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7.
I20260809 15:52:43.897136 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:43.955169 11012 raft_consensus.cc:3037] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 6
I20260809 15:52:43.955758 10931 raft_consensus.cc:1261] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 15:52:43.897169 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:43.957940 11637 raft_consensus.cc:2354] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5.
I20260809 15:52:43.876382 13822 raft_consensus.cc:2781] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:52:43.903144 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:43.960132 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 50709.7us
I20260809 15:52:43.960453 13575 raft_consensus.cc:2781] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:43.961016 13575 raft_consensus.cc:686] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:43.961305 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:43.960201 13822 raft_consensus.cc:686] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [term 7 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:43.961879 13575 consensus_queue.cc:237] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:43.963095 13822 consensus_queue.cc:237] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.5, Last appended by leader: 5, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:43.964221 10978 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:43.964897 13903 raft_consensus.cc:2741] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won
W20260809 15:52:43.592586 10737 rpcz_store.cc:269] 0809 15:52:32.190957 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:32.197309 (+ 6352us) service_pool.cc:227] Handling call
0809 15:52:34.110868 (+1913559us) spinlock_profiling.cc:243] Waited 206 ms on lock 0x7b5400303754. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478af0 00007f79694fbf2a 00007f796f44dd6c 00007f796f547c3e 00007f796d859fb4 00007f796d859f42 00007f796d859e71 00007f796d859dec 00007f796d8590a2 00007f796791d94c 00007f796791cd86
0809 15:52:43.553842 (+9442974us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 15:52:41.103268 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 15:52:41.104037 (+ 769us) write_op.cc:432] Acquiring schema lock in shared mode
0809 15:52:41.104107 (+ 70us) write_op.cc:435] Acquired schema lock
0809 15:52:41.104146 (+ 39us) tablet.cc:660] Decoding operations
0809 15:52:41.323680 (+219534us) write_op.cc:620] Acquiring the partition lock for write op
0809 15:52:41.323805 (+ 125us) write_op.cc:641] Partition lock acquired for write op
0809 15:52:41.323833 (+ 28us) tablet.cc:683] Acquiring locks for 13 operations
0809 15:52:41.325101 (+ 1268us) tablet.cc:699] Row locks acquired
0809 15:52:41.325130 (+ 29us) write_op.cc:260] PREPARE: finished
0809 15:52:41.325288 (+ 158us) write_op.cc:270] Start()
0809 15:52:41.325428 (+ 140us) write_op.cc:276] Timestamp: P: 1786290761325230 usec, L: 0
0809 15:52:41.325490 (+ 62us) op_driver.cc:331] REPLICATION: starting
0809 15:52:41.392897 (+ 67407us) log.cc:842] Serialized 5311 byte log entry
0809 15:52:41.840364 (+447467us) op_driver.cc:444] REPLICATION: finished
0809 15:52:42.051774 (+211410us) write_op.cc:301] APPLY: starting
0809 15:52:42.051931 (+ 157us) tablet.cc:1365] starting BulkCheckPresence
0809 15:52:42.052363 (+ 432us) tablet.cc:1368] finished BulkCheckPresence
0809 15:52:42.052516 (+ 153us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 15:52:42.571688 (+519172us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 15:52:42.572033 (+ 345us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=13
0809 15:52:42.572080 (+ 47us) write_op.cc:312] APPLY: finished
0809 15:52:42.572478 (+ 398us) log.cc:842] Serialized 95 byte log entry
0809 15:52:42.572662 (+ 184us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:52:42.572806 (+ 144us) write_op.cc:454] Released schema lock
0809 15:52:42.572881 (+ 75us) write_op.cc:341] FINISH: Updating metrics
0809 15:52:42.573196 (+ 315us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:52:42.573224 (+ 28us) write_op.cc:454] Released schema lock
Metrics: {"reported_tablets":17,"spinlock_wait_cycles":474770176,"child_traces":[["op",{"apply.queue_time_us":136796,"apply.run_cpu_time_us":9589,"apply.run_wall_time_us":568277,"num_ops":13,"prepare.queue_time_us":222440,"prepare.run_cpu_time_us":26297,"prepare.run_wall_time_us":414628,"raft.queue_time_us":94858,"raft.run_cpu_time_us":1929,"raft.run_wall_time_us":133787,"replication_time_us":514715,"thread_start_us":319018,"threads_started":4,"wal-append.queue_time_us":181101,"wal-append.run_cpu_time_us":15682,"wal-append.run_wall_time_us":1990213}]]}
I20260809 15:52:43.877924 13653 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:43.828286 13859 raft_consensus.cc:2726] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 15:52:43.978466 10837 raft_consensus.cc:1261] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 15:52:43.909545 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:43.914228 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:43.968566 11015 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:43.914752 11014 raft_consensus.cc:3037] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 1 FOLLOWER]: Advancing to term 3
I20260809 15:52:43.857610 13823 consensus_queue.cc:1046] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260809 15:52:43.919119 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:43.988384 10926 raft_consensus.cc:2379] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 3: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:43.919160 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
W20260809 15:52:43.936947 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 75108us
I20260809 15:52:43.991393 10927 raft_consensus.cc:2379] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 5: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:43.937217 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:43.863914 13828 consensus_queue.cc:1035] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.040s
I20260809 15:52:43.903805 13679 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:43.891994 13878 consensus_queue.cc:1035] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
I20260809 15:52:43.938374 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:43.877715 13906 raft_consensus.cc:2781] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:43.958743 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:43.965384 10978 leader_election.cc:398] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 election: Vote denied by peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) with higher term. Message: Invalid argument: T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 2. Current term is 3.
I20260809 15:52:44.004663 10978 leader_election.cc:401] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260809 15:52:44.004832 11637 raft_consensus.cc:2354] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 3. Current term is 4.
W20260809 15:52:43.964531 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 58603.3us
I20260809 15:52:43.969647 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:43.945604 11018 raft_consensus.cc:2379] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 5: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:43.971386 10922 raft_consensus.cc:2454] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:43.971828 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:43.974907 10842 raft_consensus.cc:2454] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:43.973874 11012 raft_consensus.cc:2454] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
W20260809 15:52:43.985972 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 78056.5us
I20260809 15:52:44.010774 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:43.887337 13913 consensus_queue.cc:1035] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.034s
I20260809 15:52:43.945590 13936 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:44.023468 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:44.001544 11009 raft_consensus.cc:2379] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 6: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:43.970386 10734 catalog_manager.cc:5495] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: config changed from index 6 to 7, term changed from 6 to 7, leader changed from c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) to <none>, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted, no longer has a pending config: VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1), VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2), VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 7 committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:43.938354 13522 raft_consensus.cc:2781] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:44.010767 10837 raft_consensus.cc:1261] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:43.895272 11635 raft_consensus.cc:2454] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:52:44.026719 10930 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:44.032403 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:44.029038 11013 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 10.
I20260809 15:52:44.024580 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:44.030290 10844 raft_consensus.cc:1226] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:44.034526 11641 raft_consensus.cc:2354] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 2. Current term is 3.
I20260809 15:52:43.988255 13762 raft_consensus.cc:2781] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260809 15:52:44.031299 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 93213.7us
I20260809 15:52:43.980039 13900 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:44.045899 11604 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:43.988615 13836 raft_consensus.cc:2741] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260809 15:52:44.043224 13522 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:44.062994 13936 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:44.063192 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:44.004410 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:44.069017 11014 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:44.071769 10734 catalog_manager.cc:5495] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 2 to 4, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 4 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:44.073419 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:44.007709 13689 raft_consensus.cc:2726] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) with higher term. Message: Invalid argument: T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 2. Current term is 3.
W20260809 15:52:44.076896 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 65846.1us
I20260809 15:52:44.031689 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:44.049391 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:44.041638 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:44.025467 13925 raft_consensus.cc:3037] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:44.051726 11606 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:44.053158 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:44.087381 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:44.090672 11636 raft_consensus.cc:2379] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 5: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:44.091080 10840 raft_consensus.cc:3037] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:44.075028 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:44.075390 10930 raft_consensus.cc:2354] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 5. Current term is 6.
I20260809 15:52:44.095116 10978 leader_election.cc:398] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Vote denied by peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685) with higher term. Message: Invalid argument: T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5.
I20260809 15:52:44.095726 10978 leader_election.cc:401] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260809 15:52:44.096025 11637 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 5 LEARNER]: Advancing to term 7
I20260809 15:52:44.095414 10924 raft_consensus.cc:2379] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 7: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:44.080569 10734 catalog_manager.cc:5495] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 2 to 5, leader changed from c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 5 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:44.036321 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:44.098690 11641 raft_consensus.cc:3037] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:44.102272 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:44.092147 11014 raft_consensus.cc:2454] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:44.104173 11604 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:44.098408 10734 catalog_manager.cc:5495] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:44.044809 13762 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 15:52:44.109265 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 57600.3us
I20260809 15:52:44.035898 13810 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:44.119776 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:44.063076 13522 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:44.102053 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:44.115744 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:44.126436 11009 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:44.119189 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:44.120693 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:44.130229 11641 raft_consensus.cc:2454] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:44.121552 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:44.121970 10978 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:44.133867 10843 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:44.134136 13861 raft_consensus.cc:2726] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:44.123461 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:44.022979 13679 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:44.134661 10978 leader_election.cc:398] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Vote denied by peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685) with higher term. Message: Invalid argument: T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 3. Current term is 4.
I20260809 15:52:44.135087 10839 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:44.119724 10923 raft_consensus.cc:1226] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:44.136405 13679 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:44.071136 13906 raft_consensus.cc:686] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:44.076481 13830 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:43.989066 13653 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:44.139796 10840 raft_consensus.cc:2454] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:44.141110 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:44.150807 10841 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Advancing to term 10
I20260809 15:52:44.097891 11634 raft_consensus.cc:2454] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:44.143973 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:44.118606 13873 raft_consensus.cc:2781] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:44.175542 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:44.176384 10848 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:44.124177 13819 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:44.178169 13819 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:44.179025 10922 raft_consensus.cc:1226] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260809 15:52:44.180752 13819 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:43.997860 13820 consensus_queue.cc:1035] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260809 15:52:44.171685 10930 raft_consensus.cc:2379] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 7: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:44.171267 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:44.195545 10841 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 10.
I20260809 15:52:44.145099 10927 raft_consensus.cc:2454] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:52:44.115974 13870 raft_consensus.cc:2781] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:44.119426 13689 raft_consensus.cc:2726] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685) with higher term. Message: Invalid argument: T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5.
I20260809 15:52:44.120711 13762 raft_consensus.cc:3037] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:44.143530 11637 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 7 LEARNER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:52:44.022393 13837 consensus_queue.cc:1035] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.053s
I20260809 15:52:44.110936 10734 catalog_manager.cc:5495] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 3 to 5, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2), now has a pending config: VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1), VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2), VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3), VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 5 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } health_report { overall_health: UNKNOWN } } } pending_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:44.190276 10843 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:52:44.191017 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:44.185040 10926 raft_consensus.cc:2454] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:44.140682 13900 raft_consensus.cc:2781] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:44.196024 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:44.206527 10848 raft_consensus.cc:3037] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:44.184531 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:44.197913 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 164568us
I20260809 15:52:44.204869 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:44.207787 10927 raft_consensus.cc:2379] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 5: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:44.137813 13628 consensus_queue.cc:1035] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260809 15:52:44.156347 11009 raft_consensus.cc:1261] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260809 15:52:44.220265 10923 raft_consensus.cc:2354] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 5. Current term is 6.
I20260809 15:52:44.211297 10930 raft_consensus.cc:1226] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 7. Current term is 8. Ops: []
I20260809 15:52:44.218747 10734 catalog_manager.cc:5495] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } }
I20260809 15:52:44.226418 10842 raft_consensus.cc:1226] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:44.227413 10734 catalog_manager.cc:5495] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 reported cstate change: term changed from 2 to 4. New cstate: current_term: 4 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } }
I20260809 15:52:44.199868 13893 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:44.197283 13882 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:44.150198 13917 consensus_queue.cc:1035] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.061s
I20260809 15:52:44.138114 13810 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:44.152763 13830 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:44.245080 11009 raft_consensus.cc:1226] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:44.196946 13873 raft_consensus.cc:686] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:44.210489 13870 raft_consensus.cc:686] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:44.221648 13900 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:44.255229 10927 raft_consensus.cc:1226] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:44.253412 10848 raft_consensus.cc:2454] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:44.254693 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:44.258148 10924 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:44.260006 10930 raft_consensus.cc:1261] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 15:52:44.260640 10848 raft_consensus.cc:1261] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
W20260809 15:52:44.269479 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 151826us
I20260809 15:52:44.273684 11606 leader_election.cc:302] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:44.274909 13859 raft_consensus.cc:2781] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:44.279844 13859 raft_consensus.cc:686] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:44.280611 10842 raft_consensus.cc:1226] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260809 15:52:44.280718 13859 consensus_queue.cc:237] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:44.284420 10921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 736) took 1409 ms. Trace:
I20260809 15:52:44.284818 10921 rpcz_store.cc:276] 0809 15:52:42.874758 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:43.282734 (+407976us) service_pool.cc:227] Handling call
0809 15:52:43.322472 (+ 39738us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:43.779403 (+456931us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:44.284155 (+504752us) spinlock_profiling.cc:243] Waited 456 ms on lock 0x7b540024c7d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:44.284359 (+ 204us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1049259904}
I20260809 15:52:44.286263 13810 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:44.292285 11009 raft_consensus.cc:1261] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260809 15:52:44.261428 13900 raft_consensus.cc:3037] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:44.284456 13890 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:44.296587 11634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41488 (request call id 854) took 1584 ms. Trace:
I20260809 15:52:44.298681 11632 raft_consensus.cc:1226] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:44.299162 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:44.348214 10924 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:44.348645 10848 raft_consensus.cc:1226] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260809 15:52:44.299105 13873 consensus_queue.cc:237] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:44.248369 13823 consensus_queue.cc:1035] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.027s
I20260809 15:52:44.304152 10978 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:44.305271 13653 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:44.306867 13575 consensus_queue.cc:1035] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 15:52:44.310639 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:44.312207 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:44.313050 13870 consensus_queue.cc:237] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:44.316553 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:44.356740 10927 raft_consensus.cc:3037] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:44.261791 13594 consensus_queue.cc:1046] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.059s
I20260809 15:52:44.319211 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:44.319636 13830 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:44.320654 10891 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:44.365702 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 45168.1us
I20260809 15:52:44.366456 10891 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:44.368592 10891 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:44.321652 13809 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:44.288475 13861 consensus_queue.cc:1035] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260809 15:52:44.322777 10893 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:44.324232 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:44.324149 13837 consensus_queue.cc:1046] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260809 15:52:44.328378 10839 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:44.331444 13907 raft_consensus.cc:3032] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:44.332864 13762 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:44.303328 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:44.346601 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:44.240553 13906 consensus_queue.cc:237] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:44.355189 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 51137us
I20260809 15:52:44.374815 10978 leader_election.cc:398] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 election: Vote denied by peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685) with higher term. Message: Invalid argument: T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 2. Current term is 3.
I20260809 15:52:44.375948 10978 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
W20260809 15:52:44.359369 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 255245us
I20260809 15:52:44.360584 10923 raft_consensus.cc:2379] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:44.379334 11017 raft_consensus.cc:1261] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:44.362694 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:44.385270 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:44.386070 10842 raft_consensus.cc:2454] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
W20260809 15:52:44.443548 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 120872us
I20260809 15:52:44.445945 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:44.447001 11640 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:44.448091 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:44.455847 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 82060.8us
I20260809 15:52:44.456561 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
W20260809 15:52:44.459144 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 73698.6us
I20260809 15:52:44.460223 11606 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:44.460361 10893 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:44.467697 11604 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:44.359859 13959 raft_consensus.cc:2781] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:52:44.470376 13819 raft_consensus.cc:2726] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:44.474866 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:44.371513 13890 raft_consensus.cc:2726] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:44.371645 13881 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:44.486898 10926 raft_consensus.cc:1226] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:44.489585 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:44.347683 11634 rpcz_store.cc:276] 0809 15:52:42.711872 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:42.719354 (+ 7482us) service_pool.cc:227] Handling call
0809 15:52:44.296405 (+1577051us) spinlock_profiling.cc:243] Waited 982 ms on lock 0x7b54005d2854. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:44.296510 (+ 105us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2259489024}
I20260809 15:52:44.375609 13907 raft_consensus.cc:729] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:44.490235 11640 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:44.500447 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:44.496407 10893 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:44.468215 13882 raft_consensus.cc:2781] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:44.495784 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:44.498234 13929 raft_consensus.cc:2781] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:44.525830 11009 raft_consensus.cc:1261] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
W20260809 15:52:44.526203 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 150309us
W20260809 15:52:44.526821 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 63589.5us
I20260809 15:52:44.527473 10891 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:44.446096 13867 consensus_queue.cc:1046] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.101s
I20260809 15:52:44.459967 13522 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:44.495805 13959 raft_consensus.cc:686] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 7 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:44.548026 10927 raft_consensus.cc:2454] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
I20260809 15:52:44.549817 11640 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:44.489907 13679 consensus_queue.cc:1035] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.042s
I20260809 15:52:44.477957 13936 consensus_queue.cc:1046] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260809 15:52:44.619870 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:44.487427 13893 consensus_queue.cc:1046] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260809 15:52:44.647079 10922 raft_consensus.cc:1226] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:44.647375 11639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41482 (request call id 665) took 1396 ms. Trace:
I20260809 15:52:44.647747 11639 rpcz_store.cc:276] 0809 15:52:43.250731 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:43.253064 (+ 2333us) service_pool.cc:227] Handling call
0809 15:52:44.647196 (+1394132us) spinlock_profiling.cc:243] Waited 6.97 ms on lock 0x7b540066b554. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:44.647321 (+ 125us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":16040832}
I20260809 15:52:44.648597 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:44.662333 11639 raft_consensus.cc:2379] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 3: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:44.614848 13653 raft_consensus.cc:2726] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:44.535866 13963 raft_consensus.cc:2781] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:44.676437 13893 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:44.677093 13893 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:44.626816 13628 raft_consensus.cc:2781] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:44.619730 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:44.671543 13503 consensus_meta.cc:287] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.838s user 0.010s sys 0.001s
I20260809 15:52:44.670961 13958 raft_consensus.cc:3032] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:44.578107 13917 consensus_queue.cc:1046] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.094s
W20260809 15:52:44.685601 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 189270us
I20260809 15:52:44.675136 13930 raft_consensus.cc:2781] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:44.709233 10978 leader_election.cc:398] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Vote denied by peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783) with higher term. Message: Invalid argument: T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 5. Current term is 6.
I20260809 15:52:44.709646 10978 leader_election.cc:401] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260809 15:52:44.679455 13963 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:44.683142 13628 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 15:52:44.717295 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 248064us
I20260809 15:52:44.682886 13809 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:44.721621 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 261456us
I20260809 15:52:44.726269 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:44.617116 13907 consensus_queue.cc:260] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:44.726857 13864 raft_consensus.cc:3032] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:52:44.701066 13958 raft_consensus.cc:729] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:44.729566 10978 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:44.744566 10978 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:44.721441 13628 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
W20260809 15:52:44.757685 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 230269us
I20260809 15:52:44.718533 13963 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:44.759922 10891 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:44.765702 11636 raft_consensus.cc:1226] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:44.765890 13820 raft_consensus.cc:2726] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:44.730883 13819 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:44.734292 13917 raft_consensus.cc:2726] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783) with higher term. Message: Invalid argument: T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 5. Current term is 6.
W20260809 15:52:44.748622 13961 log.cc:924] Time spent T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.132s user 0.000s sys 0.001s
I20260809 15:52:44.734577 13864 raft_consensus.cc:729] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:44.699854 13503 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:44.737691 13930 raft_consensus.cc:686] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:44.778280 13819 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:44.734305 11634 raft_consensus.cc:2454] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:44.748564 13836 raft_consensus.cc:2726] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:44.771492 13936 raft_consensus.cc:3032] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:44.734062 13959 consensus_queue.cc:237] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:44.802663 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
W20260809 15:52:44.803719 13966 log.cc:924] Time spent T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.083s user 0.000s sys 0.000s
I20260809 15:52:44.804111 13819 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:44.769577 13838 consensus_queue.cc:1046] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.036s
W20260809 15:52:44.787267 13925 consensus_meta.cc:287] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.668s user 0.004s sys 0.007s
I20260809 15:52:44.810523 11642 raft_consensus.cc:2354] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 5. Current term is 6.
I20260809 15:52:44.806283 13936 raft_consensus.cc:729] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:44.818147 13522 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:44.826242 11634 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:44.832458 13838 consensus_queue.cc:1225] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer 20730d75ad954a82b126148f8152a9f3 log is divergent from this leader: its last log entry 5.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260809 15:52:44.810221 13967 log.cc:924] Time spent T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.073s user 0.001s sys 0.000s
W20260809 15:52:44.806027 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 88854us
I20260809 15:52:44.760774 13977 consensus_queue.cc:1035] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.080s
W20260809 15:52:44.825745 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 215778us
I20260809 15:52:44.835987 13925 raft_consensus.cc:2726] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260809 15:52:44.856328 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 93082.2us
I20260809 15:52:44.821167 10978 leader_election.cc:398] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Vote denied by peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783) with higher term. Message: Invalid argument: T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 5. Current term is 6.
I20260809 15:52:44.862810 10893 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:44.864974 10978 leader_election.cc:401] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260809 15:52:44.866353 11009 raft_consensus.cc:1226] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:44.868145 10811 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:44.871338 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:44.872279 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:44.876184 11018 raft_consensus.cc:1261] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 15:52:44.877341 10893 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:44.877595 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:44.870100 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:44.878634 10811 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:44.880102 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 59100.7us
I20260809 15:52:44.879113 10893 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:44.882938 13503 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:44.883224 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:44.884320 13809 consensus_queue.cc:1035] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.005s
I20260809 15:52:44.884861 13838 raft_consensus.cc:2726] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783) with higher term. Message: Invalid argument: T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 5. Current term is 6.
I20260809 15:52:44.884639 13628 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:44.851336 13588 raft_consensus.cc:2781] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:44.885830 13989 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260809 15:52:44.885895 13825 raft_consensus.cc:2726] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:44.886439 13588 raft_consensus.cc:686] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 6 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:44.886621 13989 raft_consensus.cc:686] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 10 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:44.886806 11639 raft_consensus.cc:1226] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:44.887399 13588 consensus_queue.cc:237] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:44.887492 13989 consensus_queue.cc:237] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:44.892400 13522 raft_consensus.cc:2781] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:52:44.892947 13522 raft_consensus.cc:686] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
W20260809 15:52:44.894685 13983 log.cc:924] Time spent T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.064s user 0.000s sys 0.000s
I20260809 15:52:44.893899 13522 consensus_queue.cc:237] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:44.805790 13958 consensus_queue.cc:260] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:44.884377 13982 raft_consensus.cc:2781] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:44.896620 13982 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:44.896963 13982 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:44.898142 13628 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:44.901684 10978 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:44.894862 10893 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:44.904381 10893 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9
W20260809 15:52:44.877115 13962 log.cc:924] Time spent T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.191s user 0.001s sys 0.000s
I20260809 15:52:44.908442 13819 consensus_queue.cc:1046] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.003s
I20260809 15:52:44.906266 13979 raft_consensus.cc:2726] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:44.910635 13838 raft_consensus.cc:2781] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:44.908066 10978 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:44.907238 10893 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:44.914710 10811 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:44.960444 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 45815.3us
I20260809 15:52:44.938336 10978 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:44.961961 10927 raft_consensus.cc:1261] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 15:52:44.932384 10929 raft_consensus.cc:1226] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:44.902917 13905 raft_consensus.cc:2726] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:44.964325 10930 raft_consensus.cc:1261] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260809 15:52:44.965711 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:44.979215 10923 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
W20260809 15:52:44.976141 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 240249us
W20260809 15:52:44.974943 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 245520us
I20260809 15:52:44.976398 10811 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:44.984736 13820 raft_consensus.cc:2726] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:44.984882 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:44.985524 13820 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:44.988337 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 50089.2us
W20260809 15:52:44.991699 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 59529.8us
W20260809 15:52:44.999778 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 67447.3us
I20260809 15:52:45.000458 10811 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:45.000566 10893 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:44.931143 13929 raft_consensus.cc:686] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:44.982769 11606 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:44.931761 13838 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:44.974539 13809 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:45.006171 11009 raft_consensus.cc:3032] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:45.010470 11009 raft_consensus.cc:729] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:44.977353 13890 raft_consensus.cc:2781] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:52:44.909389 13859 consensus_queue.cc:1046] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.083s
I20260809 15:52:45.005887 10978 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:44.959532 13917 raft_consensus.cc:2726] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:44.986110 13820 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:44.851612 13930 consensus_queue.cc:237] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:45.005959 13594 raft_consensus.cc:2781] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:44.930559 13656 consensus_queue.cc:1046] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
I20260809 15:52:44.901333 13936 consensus_queue.cc:260] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.056838 10978 leader_election.cc:302] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:45.014925 11009 consensus_queue.cc:260] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.064954 11009 raft_consensus.cc:3037] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:45.017472 10811 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:45.062417 13929 consensus_queue.cc:237] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.012135 13588 raft_consensus.cc:2781] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:45.008757 13838 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:45.012001 13503 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:45.011174 13906 raft_consensus.cc:2726] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:52:44.988816 13915 consensus_queue.cc:1046] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 7, Time since last communication: 0.031s
I20260809 15:52:44.998773 13822 consensus_queue.cc:1035] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.062s
I20260809 15:52:45.077478 13820 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:45.078166 13656 raft_consensus.cc:3032] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 15:52:45.078560 13656 raft_consensus.cc:729] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:45.059826 13594 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:45.079809 13594 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:45.083472 13984 consensus_queue.cc:1035] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:52:45.083858 13820 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:45.084417 13820 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:45.013608 13980 raft_consensus.cc:2781] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:45.086436 13980 raft_consensus.cc:686] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:45.086643 13820 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:45.087364 13980 consensus_queue.cc:237] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.068554 13828 raft_consensus.cc:2726] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260809 15:52:45.088994 13853 consensus_meta.cc:287] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 1.231s user 0.002s sys 0.011s
I20260809 15:52:45.069468 10842 raft_consensus.cc:1261] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260809 15:52:45.074695 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:45.091565 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:45.092170 10839 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:45.072754 10978 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
W20260809 15:52:45.077333 13968 log.cc:924] Time spent T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.105s user 0.000s sys 0.000s
I20260809 15:52:45.079365 13656 consensus_queue.cc:260] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.063859 13917 consensus_queue.cc:1046] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260809 15:52:45.099130 13656 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
W20260809 15:52:45.088049 13881 consensus_meta.cc:287] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.585s user 0.008s sys 0.000s
I20260809 15:52:45.020252 13819 raft_consensus.cc:2726] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 15:52:45.076500 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:45.089169 13982 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:45.099628 10838 raft_consensus.cc:1261] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 8. (term mismatch)
I20260809 15:52:45.019490 13905 raft_consensus.cc:2726] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:45.109069 11009 raft_consensus.cc:1261] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260809 15:52:45.110713 11009 pending_rounds.cc:76] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa: Aborting all ops after (but not including) 5
I20260809 15:52:45.111119 11009 pending_rounds.cc:97] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260809 15:52:45.110388 10838 pending_rounds.cc:76] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536: Aborting all ops after (but not including) 7
I20260809 15:52:45.020346 13974 consensus_queue.cc:1046] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.079s
I20260809 15:52:45.111626 10838 pending_rounds.cc:97] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536: Aborting uncommitted NO_OP operation due to leader change: 6.8
I20260809 15:52:45.069806 13959 raft_consensus.cc:2726] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260809 15:52:45.112046 11009 raft_consensus.cc:2866] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:45.099464 13917 raft_consensus.cc:3032] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Stepping down as leader of term 3
W20260809 15:52:45.115971 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 115489us
I20260809 15:52:45.073757 13503 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.121809 10929 raft_consensus.cc:1261] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:45.121768 10839 raft_consensus.cc:2454] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:45.008164 13979 consensus_queue.cc:1046] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.062s
I20260809 15:52:45.126190 10838 raft_consensus.cc:2866] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:45.103791 13820 raft_consensus.cc:2781] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:45.132175 13917 raft_consensus.cc:729] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:45.153275 13820 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:45.154147 13820 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:45.154237 10920 raft_consensus.cc:1226] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:45.102885 13870 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:45.155122 13870 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.134334 13905 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:45.110569 13881 raft_consensus.cc:2726] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:45.124101 13936 raft_consensus.cc:3037] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:45.157320 13870 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:45.125087 13588 raft_consensus.cc:686] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:45.170720 10924 raft_consensus.cc:1261] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
W20260809 15:52:45.135569 13900 consensus_meta.cc:287] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.784s user 0.009s sys 0.003s
I20260809 15:52:45.126528 13985 raft_consensus.cc:482] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:45.180083 11640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 884) took 1085 ms. Trace:
I20260809 15:52:45.180496 11640 rpcz_store.cc:276] 0809 15:52:44.094756 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:44.674943 (+580187us) service_pool.cc:227] Handling call
0809 15:52:44.696085 (+ 21142us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:44.696375 (+ 290us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:44.696409 (+ 34us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:44.711273 (+ 14864us) log.cc:842] Serialized 26 byte log entry
0809 15:52:44.934617 (+223344us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:52:44.934665 (+ 48us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:44.934750 (+ 85us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:45.179859 (+245109us) raft_consensus.cc:1671] finished
0809 15:52:45.179881 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:45.180024 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"thread_start_us":192802,"threads_started":1,"wal-append.queue_time_us":437748}
I20260809 15:52:45.182246 13998 raft_consensus.cc:3032] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:45.182372 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:45.182705 13998 raft_consensus.cc:729] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:45.182988 11640 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:45.183521 13998 consensus_queue.cc:260] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.184533 13998 raft_consensus.cc:3037] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:45.188953 11634 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 8.
I20260809 15:52:45.207872 11640 raft_consensus.cc:2454] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:45.196411 13869 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:45.195979 13900 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.197265 13679 raft_consensus.cc:3032] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:45.205351 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:45.205626 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 77845.4us
I20260809 15:52:45.225399 10926 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:45.185668 10923 raft_consensus.cc:2454] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:45.222680 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:45.227170 10978 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:45.193637 11633 raft_consensus.cc:2454] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:45.230082 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:45.239470 10931 raft_consensus.cc:1226] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260809 15:52:45.240597 10930 raft_consensus.cc:1226] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260809 15:52:45.244550 11641 raft_consensus.cc:1226] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: [4.7-4.7]
I20260809 15:52:45.244879 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:45.246580 10893 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:45.249106 11011 raft_consensus.cc:2379] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 5: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:45.248047 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:45.157289 13905 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.228828 13679 raft_consensus.cc:729] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:45.227619 13992 raft_consensus.cc:2741] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260809 15:52:45.264520 14000 raft_consensus.cc:2781] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:45.266937 13809 raft_consensus.cc:2726] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:45.236613 13869 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.267657 14005 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:45.269062 13679 consensus_queue.cc:260] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.270540 11634 raft_consensus.cc:1226] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:45.228673 13594 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:45.272035 14004 consensus_queue.cc:1035] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260809 15:52:45.272492 13946 log.cc:924] Time spent T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 1.156s user 0.008s sys 0.005s
I20260809 15:52:45.272831 13588 consensus_queue.cc:237] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.265858 13503 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:45.273510 13820 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.274653 13959 consensus_queue.cc:1046] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260809 15:52:45.278550 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 267644us
I20260809 15:52:45.228796 13977 consensus_queue.cc:1035] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.033s
I20260809 15:52:45.254542 13522 raft_consensus.cc:2781] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election won for term 8
W20260809 15:52:45.310832 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 84330.7us
I20260809 15:52:45.219606 13985 raft_consensus.cc:504] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.329560 11640 raft_consensus.cc:1799] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Handling vote request from an unknown peer d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:45.290731 14000 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:45.336201 11640 raft_consensus.cc:2396] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for term 8 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260809 15:52:45.335520 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 318065us
W20260809 15:52:45.340262 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 109792us
I20260809 15:52:45.220189 13917 consensus_queue.cc:260] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.329137 13679 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:45.379738 10811 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:45.382023 11012 raft_consensus.cc:1226] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:45.383613 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:45.383915 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:45.384410 10930 raft_consensus.cc:2362] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6. Re-sending same reply.
I20260809 15:52:45.392124 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:45.339563 13982 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:45.341398 13959 consensus_queue.cc:1225] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer 20730d75ad954a82b126148f8152a9f3 log is divergent from this leader: its last log entry 7.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260809 15:52:45.341977 11634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 887) took 1188 ms. Trace:
I20260809 15:52:45.341091 14000 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
W20260809 15:52:45.449661 13963 consensus_meta.cc:287] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.671s user 0.012s sys 0.000s
I20260809 15:52:45.389467 14019 raft_consensus.cc:2781] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election won for term 6
W20260809 15:52:45.583752 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 211266us
I20260809 15:52:45.444474 13825 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:45.620322 11606 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:45.320791 14005 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.487138 13809 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:52:45.721032 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 100789us
I20260809 15:52:45.727506 14004 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:45.728415 14004 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.729900 14005 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:45.488517 13656 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:45.730706 13977 consensus_queue.cc:1035] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260809 15:52:45.733420 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 348796us
I20260809 15:52:45.734987 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:45.735159 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:45.731464 13656 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.753702 13656 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:45.755362 13656 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:45.756959 13656 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.763092 13656 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:45.448321 11634 rpcz_store.cc:276] 0809 15:52:44.154056 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:45.247590 (+1093534us) service_pool.cc:227] Handling call
0809 15:52:45.270298 (+ 22708us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:45.327626 (+ 57328us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:45.338805 (+ 11179us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:52:45.736003 13964 log.cc:924] Time spent T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.790s user 0.005s sys 0.003s
I20260809 15:52:45.656241 13825 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.786973 13656 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:45.628116 13963 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.723620 13822 consensus_queue.cc:1035] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.062s
I20260809 15:52:45.722332 13809 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:45.779357 13905 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:45.793176 13594 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:45.813738 11642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41478 (request call id 971) took 1927 ms (client timeout 2066 ms). Trace:
W20260809 15:52:45.814666 11642 rpcz_store.cc:269] 0809 15:52:43.885876 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:43.961037 (+ 75161us) service_pool.cc:227] Handling call
0809 15:52:45.810918 (+1849881us) spinlock_profiling.cc:243] Waited 843 ms on lock 0x7b540066fdd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:45.813662 (+ 2744us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1939929472}
W20260809 15:52:45.818311 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 90256.7us
I20260809 15:52:45.788583 13869 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:45.820243 14004 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:45.821282 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:45.800688 13820 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:45.824893 11606 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:45.836549 11642 raft_consensus.cc:1226] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260809 15:52:45.810163 14028 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:45.801884 13656 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:45.838642 11642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 674) took 1470 ms. Trace:
I20260809 15:52:45.845310 11642 rpcz_store.cc:276] 0809 15:52:44.367824 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:45.822902 (+1455078us) service_pool.cc:227] Handling call
0809 15:52:45.836239 (+ 13337us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:45.838358 (+ 2119us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:45.838580 (+ 222us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:45.814445 13905 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:45.841449 11606 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:45.842247 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:45.843066 11637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 883) took 1775 ms. Trace:
I20260809 15:52:45.825546 11634 raft_consensus.cc:1261] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260809 15:52:45.847363 11017 raft_consensus.cc:2379] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 2: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:45.846750 13905 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:45.848006 11634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 673) took 1480 ms. Trace:
I20260809 15:52:45.846763 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:45.848596 11634 rpcz_store.cc:276] 0809 15:52:44.367472 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:45.792796 (+1425324us) service_pool.cc:227] Handling call
0809 15:52:45.799202 (+ 6406us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:45.847598 (+ 48396us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:45.847933 (+ 335us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:45.850710 11642 raft_consensus.cc:1226] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260809 15:52:45.851472 11642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 678) took 1482 ms. Trace:
I20260809 15:52:45.852054 11642 rpcz_store.cc:276] 0809 15:52:44.369062 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:45.849813 (+1480751us) service_pool.cc:227] Handling call
0809 15:52:45.850403 (+ 590us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:45.851195 (+ 792us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:45.851415 (+ 220us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:45.847059 11637 rpcz_store.cc:276] 0809 15:52:44.068028 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:44.201478 (+133450us) service_pool.cc:227] Handling call
0809 15:52:44.201772 (+ 294us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:44.229374 (+ 27602us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:44.229421 (+ 47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:44.242046 (+ 12625us) log.cc:842] Serialized 26 byte log entry
0809 15:52:44.501697 (+259651us) raft_consensus.cc:1631] Marking committed up to 7
0809 15:52:44.501737 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:44.501801 (+ 64us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:45.276404 (+774603us) spinlock_profiling.cc:243] Waited 90.6 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:52:45.837298 (+560894us) raft_consensus.cc:1671] finished
0809 15:52:45.837329 (+ 31us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:45.842988 (+ 5659us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":91,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":10615,"spinlock_wait_cycles":208449408,"thread_start_us":227803,"threads_started":1,"wal-append.queue_time_us":272169}
I20260809 15:52:45.853875 11642 raft_consensus.cc:1226] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260809 15:52:45.854697 11642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 680) took 1485 ms. Trace:
I20260809 15:52:45.855135 11637 raft_consensus.cc:1226] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:45.855242 13905 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:45.856467 11637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 889) took 1146 ms. Trace:
I20260809 15:52:45.858300 11637 rpcz_store.cc:276] 0809 15:52:44.710029 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:45.853431 (+1143402us) service_pool.cc:227] Handling call
0809 15:52:45.854037 (+ 606us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:45.856189 (+ 2152us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:45.856411 (+ 222us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:45.859731 13809 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:45.864239 14029 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:45.864399 14013 raft_consensus.cc:2726] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 15:52:45.865481 11635 raft_consensus.cc:1226] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:45.865754 14029 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:45.867194 14032 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:45.867976 14032 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:45.870410 10978 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.626s user 0.005s sys 0.018s
W20260809 15:52:45.870831 10978 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.627s user 0.005s sys 0.019s
W20260809 15:52:45.871204 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 627155us
I20260809 15:52:45.855080 11642 rpcz_store.cc:276] 0809 15:52:44.369658 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:45.853010 (+1483352us) service_pool.cc:227] Handling call
0809 15:52:45.853604 (+ 594us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:45.854388 (+ 784us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:45.854643 (+ 255us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:45.874590 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:45.875387 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:45.876041 10926 raft_consensus.cc:2379] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:45.894012 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:45.894857 10842 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:45.837083 14006 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:45.897068 11018 raft_consensus.cc:3032] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:45.900388 10839 raft_consensus.cc:1261] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260809 15:52:45.902118 11018 raft_consensus.cc:729] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:45.903960 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:45.898257 10842 raft_consensus.cc:1261] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260809 15:52:45.906925 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:45.909384 10930 raft_consensus.cc:2396] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260809 15:52:45.904734 10926 raft_consensus.cc:2454] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 7.
I20260809 15:52:45.913395 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:45.916128 14029 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:45.916285 13820 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:45.917095 13820 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:45.903239 11018 consensus_queue.cc:260] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.920989 11018 raft_consensus.cc:3037] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
W20260809 15:52:45.922199 14018 log.cc:924] Time spent T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.180s user 0.000s sys 0.001s
I20260809 15:52:45.924046 11011 raft_consensus.cc:1261] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260809 15:52:45.926641 14031 consensus_queue.cc:1046] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 15:52:45.929569 14031 raft_consensus.cc:3032] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:45.931289 13825 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:45.931833 10719 catalog_manager.cc:5025] ChangeConfig:REMOVE_PEER RPC for tablet 0b2b0df042e44725a7c91deae3df1956 with cas_config_opid_index 4: ChangeConfig:REMOVE_PEER failed with leader d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223) due to error UNKNOWN_ERROR; will retry: Illegal state: Replica d74944fa9fa14ec78b0640ea709ff3aa is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 10 committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:45.844759 13985 leader_election.cc:288] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:45.935714 11642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 891) took 1224 ms. Trace:
I20260809 15:52:45.936205 11642 rpcz_store.cc:276] 0809 15:52:44.711638 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:45.873312 (+1161674us) service_pool.cc:227] Handling call
0809 15:52:45.935146 (+ 61834us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:45.935364 (+ 218us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:45.935395 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:45.935445 (+ 50us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:52:45.935486 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:45.935547 (+ 61us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:45.935657 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:45.945824 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:45.947731 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:45.948709 14027 log.cc:924] Time spent T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.111s user 0.001s sys 0.000s
I20260809 15:52:45.950526 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:45.953325 13838 consensus_meta.cc:287] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.852s user 0.009s sys 0.004s
I20260809 15:52:45.973534 11018 raft_consensus.cc:1261] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260809 15:52:45.974112 11018 pending_rounds.cc:76] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa: Aborting all ops after (but not including) 5
I20260809 15:52:45.927464 13820 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:45.974378 11018 pending_rounds.cc:97] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260809 15:52:45.975181 11018 raft_consensus.cc:2866] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260809 15:52:45.977727 13998 consensus_meta.cc:287] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.793s user 0.004s sys 0.011s
W20260809 15:52:45.980161 13936 consensus_meta.cc:287] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.812s user 0.012s sys 0.000s
I20260809 15:52:45.900995 13869 raft_consensus.cc:2781] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election won for term 5
W20260809 15:52:45.945518 13679 consensus_meta.cc:287] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.555s user 0.013s sys 0.000s
I20260809 15:52:45.917095 14029 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:45.987393 10930 raft_consensus.cc:1261] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260809 15:52:45.940680 13963 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:45.934640 14031 raft_consensus.cc:729] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:45.990798 11642 raft_consensus.cc:1226] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 7 LEARNER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:45.998786 13656 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:45.871497 14028 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:45.973260 13985 consensus_queue.cc:1035] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.027s
I20260809 15:52:45.953855 13838 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:46.056362 13867 raft_consensus.cc:3032] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Stepping down as leader of term 1
I20260809 15:52:46.053844 10844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45578 (request call id 1997) took 2149 ms. Trace:
I20260809 15:52:46.090427 10844 rpcz_store.cc:276] 0809 15:52:43.904091 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:43.918574 (+ 14483us) service_pool.cc:227] Handling call
0809 15:52:44.006826 (+ 88252us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:44.036086 (+ 29260us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:46.053583 (+2017497us) spinlock_profiling.cc:243] Waited 37.2 ms on lock 0x7b540037ebd0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:46.053717 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":85648768}
I20260809 15:52:46.055166 13837 raft_consensus.cc:3032] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:46.078743 10848 raft_consensus.cc:1226] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260809 15:52:46.078626 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:46.079478 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:46.103255 10847 raft_consensus.cc:2396] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260809 15:52:46.081067 10924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54236 (request call id 2022) took 1774 ms. Trace:
I20260809 15:52:46.103636 10924 rpcz_store.cc:276] 0809 15:52:44.306317 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:44.456118 (+149801us) service_pool.cc:227] Handling call
0809 15:52:44.487027 (+ 30909us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:45.171742 (+684715us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:46.080777 (+909035us) spinlock_profiling.cc:243] Waited 680 ms on lock 0x7b540018b354. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:46.080995 (+ 218us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1565028608}
I20260809 15:52:46.084715 10930 pending_rounds.cc:76] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3: Aborting all ops after (but not including) 5
I20260809 15:52:46.084807 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:46.086374 10840 raft_consensus.cc:1226] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260809 15:52:46.067029 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:46.105887 11642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41478 (request call id 983) took 1080 ms. Trace:
I20260809 15:52:46.052956 13762 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:45.984826 13820 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:46.098755 13809 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:46.019919 13869 raft_consensus.cc:686] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:46.000869 14032 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:46.098783 11633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41478 (request call id 980) took 2205 ms (client timeout 1633 ms). Trace:
I20260809 15:52:46.099593 13838 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:46.106456 10930 pending_rounds.cc:97] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260809 15:52:46.142763 10930 raft_consensus.cc:2866] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:46.106302 11018 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:46.143766 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:46.106312 11642 rpcz_store.cc:276] 0809 15:52:45.025328 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:45.958201 (+932873us) service_pool.cc:227] Handling call
0809 15:52:45.990573 (+ 32372us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.105595 (+115022us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:46.105821 (+ 226us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:52:46.000967 14016 log.cc:924] Time spent T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.345s user 0.002s sys 0.000s
I20260809 15:52:46.118544 11011 raft_consensus.cc:1261] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 6. (index mismatch)
I20260809 15:52:46.121652 13820 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:46.116801 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:46.143175 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:46.090147 13867 raft_consensus.cc:729] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:46.150803 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:46.135944 10843 raft_consensus.cc:1226] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260809 15:52:46.161029 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:46.110762 13855 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:46.105640 13837 raft_consensus.cc:729] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:46.113615 13905 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:46.146035 10847 raft_consensus.cc:2932] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Committing config change with OpId 3.5: config changed from index 4 to 5, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:46.179283 11642 raft_consensus.cc:1226] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:46.180101 11642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41478 (request call id 984) took 1154 ms. Trace:
I20260809 15:52:46.136972 13798 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:46.182040 11642 rpcz_store.cc:276] 0809 15:52:45.025672 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.147706 (+1122034us) service_pool.cc:227] Handling call
0809 15:52:46.179059 (+ 31353us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.179836 (+ 777us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:46.180037 (+ 201us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:46.176862 14031 consensus_queue.cc:260] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.188060 14031 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:46.106590 14029 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:46.190021 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:46.108786 13859 consensus_queue.cc:1035] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.060s
I20260809 15:52:46.149420 14028 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:46.198684 13985 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:46.161028 13838 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:46.236567 11018 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:46.243260 10848 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:52:46.248468 11642 raft_consensus.cc:1261] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260809 15:52:46.249279 11642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 682) took 1223 ms. Trace:
I20260809 15:52:46.249733 11642 rpcz_store.cc:276] 0809 15:52:45.025985 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.183486 (+1157501us) service_pool.cc:227] Handling call
0809 15:52:46.248156 (+ 64670us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.248982 (+ 826us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:46.249201 (+ 219us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:46.136598 13809 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.269690 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:46.251470 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:46.251228 13867 consensus_queue.cc:260] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.282822 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
W20260809 15:52:46.250972 10929 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.801s user 0.004s sys 0.022s
W20260809 15:52:46.285377 10929 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.835s user 0.006s sys 0.023s
I20260809 15:52:46.291102 10839 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 8.
I20260809 15:52:46.225899 13762 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:46.294574 10893 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.84.1:46615 timed out after 1.555s (SENT) [suppressed 20 similar messages]
I20260809 15:52:46.296391 10848 raft_consensus.cc:2379] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 7: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
W20260809 15:52:46.136871 11633 rpcz_store.cc:269] 0809 15:52:43.893406 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:43.920397 (+ 26991us) service_pool.cc:227] Handling call
0809 15:52:46.098489 (+2178092us) spinlock_profiling.cc:243] Waited 817 ms on lock 0x7b54004f3dd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:46.098648 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1878292224}
I20260809 15:52:46.314486 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:46.180774 13905 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:46.173194 13855 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.335301 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:46.339636 11018 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:46.341158 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:46.342389 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:46.343011 11018 raft_consensus.cc:2454] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:46.343523 11604 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:46.247433 13820 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:46.244839 14028 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:46.195667 13798 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.302170 14011 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:46.371285 10848 raft_consensus.cc:1261] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 15:52:46.377084 11642 raft_consensus.cc:1261] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:52:46.246809 13985 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.387337 10843 raft_consensus.cc:1261] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 5 index: 4. (index mismatch)
I20260809 15:52:46.349071 13473 raft_consensus.cc:2781] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:46.255381 13838 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.364722 13867 raft_consensus.cc:3037] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:46.397461 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:46.398187 11018 raft_consensus.cc:2454] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:46.399492 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:46.367667 13920 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:46.399819 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:46.404426 13473 raft_consensus.cc:686] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:46.415131 13473 consensus_queue.cc:237] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.389082 14023 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:46.333336 13762 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.389662 13859 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:46.251013 13869 consensus_queue.cc:237] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.284035 13837 consensus_queue.cc:260] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.424624 13905 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:46.377959 13820 consensus_queue.cc:1046] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.040s
I20260809 15:52:46.426656 10848 raft_consensus.cc:1261] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260809 15:52:46.426517 13809 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:46.427613 11633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 685) took 1400 ms. Trace:
I20260809 15:52:46.430850 11011 raft_consensus.cc:2454] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:46.377759 14028 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.498804 13855 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:46.500885 13798 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:46.503129 14037 log.cc:924] Time spent T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.082s user 0.001s sys 0.000s
W20260809 15:52:46.522379 10891 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.783s user 0.037s sys 0.054s
I20260809 15:52:46.507962 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:46.429643 13838 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:46.504019 14029 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:46.523643 10891 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.785s user 0.037s sys 0.054s
I20260809 15:52:46.530396 11606 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:46.531570 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:46.535360 11018 raft_consensus.cc:2454] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:46.536767 11604 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:46.561964 10842 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.578s user 0.017s sys 0.052s
I20260809 15:52:46.517493 13837 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:46.571308 10809 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:46.389194 14011 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.559517 13838 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
W20260809 15:52:46.571846 10842 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.588s user 0.018s sys 0.052s
W20260809 15:52:46.530581 14000 consensus_meta.cc:287] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.961s user 0.008s sys 0.008s
I20260809 15:52:46.563006 13855 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:46.514726 13656 consensus_queue.cc:1035] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.082s
I20260809 15:52:46.421226 13920 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.436883 13859 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.536468 13913 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:52:46.594628 13522 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.187s user 0.025s sys 0.013s
W20260809 15:52:46.577931 14006 consensus_meta.cc:287] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.671s user 0.011s sys 0.005s
I20260809 15:52:46.534402 13820 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:46.571004 14029 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:46.625070 13798 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:46.641348 10976 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.692s user 0.017s sys 0.026s
W20260809 15:52:46.660742 10976 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.779s user 0.018s sys 0.031s
W20260809 15:52:46.641963 11603 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.690s user 0.012s sys 0.032s
W20260809 15:52:46.665793 11603 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.783s user 0.012s sys 0.038s
W20260809 15:52:46.604954 13929 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.882s user 0.018s sys 0.030s
I20260809 15:52:46.596416 13594 consensus_queue.cc:1035] T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.065s
I20260809 15:52:46.634966 13855 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:46.660508 13522 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.281s user 0.025s sys 0.017s
I20260809 15:52:46.630059 14039 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:46.641755 13762 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:46.510610 11633 rpcz_store.cc:276] 0809 15:52:45.026879 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.346316 (+1319437us) service_pool.cc:227] Handling call
0809 15:52:46.392105 (+ 45789us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.416794 (+ 24689us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:46.419091 (+ 2297us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:46.427111 (+ 8020us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:46.427214 (+ 103us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:46.427333 (+ 119us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:46.427530 (+ 197us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:46.609670 13838 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.688524 10923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1974) took 1305 ms. Trace:
I20260809 15:52:46.667194 14006 raft_consensus.cc:2726] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:46.693288 10923 rpcz_store.cc:276] 0809 15:52:45.383215 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:45.383342 (+ 127us) service_pool.cc:227] Handling call
0809 15:52:45.383745 (+ 403us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:45.739351 (+355606us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:45.739394 (+ 43us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:45.747334 (+ 7940us) log.cc:842] Serialized 26 byte log entry
0809 15:52:45.837806 (+ 90472us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:52:45.897061 (+ 59255us) log.cc:842] Serialized 14 byte log entry
0809 15:52:45.897288 (+ 227us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:45.897369 (+ 81us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:46.688261 (+790892us) raft_consensus.cc:1671] finished
0809 15:52:46.688287 (+ 26us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:46.688454 (+ 167us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":38035,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":35,"raft.queue_time_us":86396,"raft.run_cpu_time_us":188,"raft.run_wall_time_us":189,"thread_start_us":406997,"threads_started":2,"wal-append.queue_time_us":557272}
I20260809 15:52:46.437191 14023 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.699373 11637 raft_consensus.cc:1226] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:46.700945 13838 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:46.709807 11633 raft_consensus.cc:1226] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:46.716410 14028 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:46.716981 10922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54236 (request call id 2014) took 2534 ms. Trace:
I20260809 15:52:46.717350 14028 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:46.691731 14038 consensus_queue.cc:1035] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.289s
I20260809 15:52:46.705722 13762 raft_consensus.cc:2781] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:46.707396 13893 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:46.681284 14043 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:46.708535 13855 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:46.709779 13890 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:46.711072 13859 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:46.712392 13951 log.cc:924] Time spent T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 2.449s user 0.007s sys 0.009s
I20260809 15:52:46.736308 13857 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:46.713217 14023 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:46.717487 11633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 893) took 1465 ms. Trace:
I20260809 15:52:46.748328 13857 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:46.748898 11633 rpcz_store.cc:276] 0809 15:52:45.251830 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.705928 (+1454098us) service_pool.cc:227] Handling call
0809 15:52:46.709512 (+ 3584us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.716960 (+ 7448us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:46.717400 (+ 440us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:46.737475 14028 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.718636 11632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41478 (request call id 965) took 2834 ms. Trace:
I20260809 15:52:46.751463 11632 rpcz_store.cc:276] 0809 15:52:43.883942 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:43.886524 (+ 2582us) service_pool.cc:227] Handling call
0809 15:52:44.298514 (+411990us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:44.351975 (+ 53461us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:46.718247 (+2366272us) spinlock_profiling.cc:243] Waited 387 ms on lock 0x7b54005d2850. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:46.718511 (+ 264us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":890263424}
I20260809 15:52:46.752854 14028 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:46.692546 13985 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:46.719776 11636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41478 (request call id 981) took 2813 ms. Trace:
I20260809 15:52:46.754632 13890 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:46.754768 11636 rpcz_store.cc:276] 0809 15:52:43.906834 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:44.120839 (+214005us) service_pool.cc:227] Handling call
0809 15:52:44.150064 (+ 29225us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:44.766175 (+616111us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:46.719520 (+1953345us) spinlock_profiling.cc:243] Waited 615 ms on lock 0x7b54004e0a54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:46.719712 (+ 192us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1415288192}
I20260809 15:52:46.665607 13913 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.723143 13894 consensus_queue.cc:1035] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:46.717433 10922 rpcz_store.cc:276] 0809 15:52:44.182998 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:44.195748 (+ 12750us) service_pool.cc:227] Handling call
0809 15:52:44.212815 (+ 17067us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:44.647502 (+434687us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:46.716673 (+2069171us) spinlock_profiling.cc:243] Waited 434 ms on lock 0x7b540021a7d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:46.716865 (+ 192us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":998209408}
W20260809 15:52:46.670521 13929 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.951s user 0.018s sys 0.030s
I20260809 15:52:46.677599 13522 raft_consensus.cc:686] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 8 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:46.666779 14029 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:46.665350 13798 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.721103 10838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2206) took 3477 ms. Trace:
I20260809 15:52:46.757400 11636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 896) took 1471 ms. Trace:
I20260809 15:52:46.804176 10838 rpcz_store.cc:276] 0809 15:52:43.244048 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:43.244215 (+ 167us) service_pool.cc:227] Handling call
0809 15:52:43.244572 (+ 357us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:45.153072 (+1908500us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:46.720835 (+1567763us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b540022d654. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:46.721034 (+ 199us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4242887040}
I20260809 15:52:46.702806 14011 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:46.668203 13820 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.630879 14000 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:46.753628 14028 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:46.804319 11636 rpcz_store.cc:276] 0809 15:52:45.286145 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.756397 (+1470252us) service_pool.cc:227] Handling call
0809 15:52:46.756823 (+ 426us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.757047 (+ 224us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:46.757072 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:46.757125 (+ 53us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:46.757162 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:46.757225 (+ 63us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:46.757347 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:46.757390 14006 raft_consensus.cc:2781] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260809 15:52:46.808382 10810 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.077s user 0.046s sys 0.076s
I20260809 15:52:46.736351 13762 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:46.818850 10931 raft_consensus.cc:1226] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
W20260809 15:52:46.757596 13915 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.680s user 0.037s sys 0.029s
W20260809 15:52:46.827538 10810 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.096s user 0.046s sys 0.077s
I20260809 15:52:46.765462 13905 raft_consensus.cc:2781] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260809 15:52:46.835637 11009 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.069s user 0.004s sys 0.021s
W20260809 15:52:46.858261 11009 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.092s user 0.004s sys 0.023s
I20260809 15:52:46.829054 14006 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:46.807566 13656 raft_consensus.cc:482] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:52:46.825158 13925 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.739s user 0.034s sys 0.065s
I20260809 15:52:46.869689 11015 raft_consensus.cc:1226] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:46.872236 11636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 897) took 1062 ms. Trace:
I20260809 15:52:46.872742 11636 rpcz_store.cc:276] 0809 15:52:45.809613 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.830053 (+1020440us) service_pool.cc:227] Handling call
0809 15:52:46.871636 (+ 41583us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.871850 (+ 214us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:46.871886 (+ 36us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:46.871936 (+ 50us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:46.871978 (+ 42us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:46.872039 (+ 61us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:46.872171 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:52:46.797742 14019 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.015s user 0.025s sys 0.090s
I20260809 15:52:46.828126 13762 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:46.694278 14039 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:46.831061 13915 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.752s user 0.037s sys 0.029s
I20260809 15:52:46.857197 13905 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:52:46.875810 13925 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.789s user 0.038s sys 0.066s
I20260809 15:52:46.950996 11636 raft_consensus.cc:1261] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 7 LEARNER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260809 15:52:46.955253 11636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 898) took 1014 ms. Trace:
I20260809 15:52:46.955732 11636 rpcz_store.cc:276] 0809 15:52:45.940554 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.880774 (+940220us) service_pool.cc:227] Handling call
0809 15:52:46.950623 (+ 69849us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.954850 (+ 4227us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:46.955188 (+ 338us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:46.869792 13838 raft_consensus.cc:482] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:52:46.880366 14019 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.144s user 0.026s sys 0.095s
I20260809 15:52:46.874575 14038 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:46.830694 14028 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:46.868667 14006 raft_consensus.cc:3037] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:46.951608 13905 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:46.996896 11636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 899) took 1047 ms. Trace:
W20260809 15:52:46.996251 10847 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.663s user 0.017s sys 0.074s
W20260809 15:52:47.002140 10847 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.669s user 0.017s sys 0.074s
W20260809 15:52:46.997133 11010 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.608s user 0.012s sys 0.019s
W20260809 15:52:47.003073 11010 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.614s user 0.012s sys 0.019s
W20260809 15:52:46.997040 11014 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.896s user 0.011s sys 0.037s
W20260809 15:52:47.003845 11014 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.903s user 0.011s sys 0.037s
I20260809 15:52:46.901460 13971 consensus_queue.cc:1035] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.280s
I20260809 15:52:46.879923 13656 raft_consensus.cc:504] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:46.973361 14019 raft_consensus.cc:686] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
W20260809 15:52:46.978065 13951 log.cc:924] Time spent T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.096s user 0.000s sys 0.000s
I20260809 15:52:47.021580 10927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1969) took 2148 ms. Trace:
W20260809 15:52:46.996917 10811 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.429s user 0.000s sys 0.030s
W20260809 15:52:47.184638 10811 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.617s user 0.000s sys 0.031s
W20260809 15:52:47.185153 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 1.67564e+06us
I20260809 15:52:47.187093 10811 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:47.047746 10975 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.387s user 0.017s sys 0.017s
I20260809 15:52:46.757175 14043 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:47.037550 10927 rpcz_store.cc:276] 0809 15:52:44.872755 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:44.879301 (+ 6546us) service_pool.cc:227] Handling call
0809 15:52:44.915004 (+ 35703us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:44.972074 (+ 57070us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:47.021251 (+2049177us) spinlock_profiling.cc:243] Waited 46.6 ms on lock 0x7b540024dbd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:47.021513 (+ 262us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":107102464}
I20260809 15:52:46.994464 14047 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:46.977413 13838 raft_consensus.cc:504] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:47.190793 10975 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.530s user 0.017s sys 0.020s
W20260809 15:52:47.192920 10926 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.668s user 0.000s sys 0.018s
I20260809 15:52:46.998054 14038 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:47.059653 13920 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:47.219893 10926 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.695s user 0.000s sys 0.018s
I20260809 15:52:47.218580 14047 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:47.190454 14036 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.130s user 0.010s sys 0.019s
W20260809 15:52:47.222754 11606 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.650s user 0.018s sys 0.024s
I20260809 15:52:47.226745 11009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 613) took 1504 ms. Trace:
I20260809 15:52:47.249576 11009 rpcz_store.cc:276] 0809 15:52:45.722810 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:45.722961 (+ 151us) service_pool.cc:227] Handling call
0809 15:52:45.766125 (+ 43164us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:46.858735 (+1092610us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:46.858779 (+ 44us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:46.866056 (+ 7277us) log.cc:842] Serialized 26 byte log entry
0809 15:52:46.866260 (+ 204us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:46.866306 (+ 46us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:46.866394 (+ 88us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:47.226543 (+360149us) raft_consensus.cc:1671] finished
0809 15:52:47.226563 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:47.226689 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":149393,"raft.run_cpu_time_us":161,"raft.run_wall_time_us":161,"thread_start_us":1092073,"threads_started":1}
W20260809 15:52:47.157236 14035 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.815s user 0.020s sys 0.010s
W20260809 15:52:47.251817 11606 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.679s user 0.019s sys 0.025s
I20260809 15:52:46.999054 13522 consensus_queue.cc:237] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:47.296932 10848 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s user 0.024s sys 0.030s
W20260809 15:52:47.299222 10848 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.573s user 0.024s sys 0.030s
W20260809 15:52:47.299233 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 726491us
I20260809 15:52:47.216051 13857 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:47.301448 11606 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:47.251332 14036 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.191s user 0.010s sys 0.019s
W20260809 15:52:47.314381 10839 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.685s user 0.005s sys 0.021s
I20260809 15:52:47.229228 13810 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:47.317443 10978 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.964s user 0.013s sys 0.015s
W20260809 15:52:47.317624 10839 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.688s user 0.005s sys 0.021s
I20260809 15:52:47.238065 13798 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:47.296953 14035 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.955s user 0.021s sys 0.010s
W20260809 15:52:47.326277 10978 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.973s user 0.013s sys 0.015s
W20260809 15:52:47.326956 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 973689us
I20260809 15:52:47.311766 14000 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:47.309357 14048 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:47.320544 13857 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:47.316756 14029 leader_election.cc:288] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:47.319986 13894 raft_consensus.cc:2726] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
W20260809 15:52:47.340895 13965 log.cc:924] Time spent T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 2.816s user 0.007s sys 0.016s
I20260809 15:52:47.350564 11606 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:47.539458 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 188959us
I20260809 15:52:47.540400 11606 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
W20260809 15:52:47.334667 10978 leader_election.cc:334] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 2.066s (SENT)
W20260809 15:52:47.353564 11603 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.676s user 0.004s sys 0.015s
W20260809 15:52:47.541703 11603 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.870s user 0.005s sys 0.015s
W20260809 15:52:47.353899 10844 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.161s user 0.000s sys 0.020s
W20260809 15:52:47.542888 10844 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.350s user 0.000s sys 0.020s
W20260809 15:52:47.354379 10929 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.963s user 0.009s sys 0.010s
W20260809 15:52:47.543525 10929 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.152s user 0.010s sys 0.010s
W20260809 15:52:47.247813 11604 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.653s user 0.005s sys 0.038s
W20260809 15:52:47.354210 14025 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.373s user 0.006s sys 0.026s
W20260809 15:52:47.458312 10893 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.109s user 0.004s sys 0.044s
W20260809 15:52:47.549404 10893 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.200s user 0.004s sys 0.044s
W20260809 15:52:47.549715 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 1.25508e+06us
I20260809 15:52:47.321198 14052 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:47.463717 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 276708us
I20260809 15:52:47.329797 13810 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:47.447935 13882 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 2.834s user 0.023s sys 0.015s
I20260809 15:52:47.371866 14039 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:47.487015 10809 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.831s user 0.021s sys 0.066s
I20260809 15:52:47.491643 11640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41478 (request call id 985) took 2420 ms. Trace:
I20260809 15:52:47.493551 11642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 683) took 2467 ms. Trace:
I20260809 15:52:47.496016 13982 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:47.499285 11015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2084) took 1389 ms. Trace:
W20260809 15:52:47.505846 13978 log.cc:924] Time spent T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 2.389s user 0.000s sys 0.005s
I20260809 15:52:47.508739 11641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 879) took 3515 ms. Trace:
I20260809 15:52:47.510164 10920 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54236 (request call id 2025) took 2603 ms. Trace:
I20260809 15:52:47.513288 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1978) took 2128 ms. Trace:
I20260809 15:52:47.517571 10930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 760) took 1643 ms. Trace:
I20260809 15:52:47.518687 11017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2083) took 1416 ms. Trace:
W20260809 15:52:47.520779 13947 log.cc:924] Time spent T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 3.342s user 0.021s sys 0.008s
W20260809 15:52:47.544063 11604 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.971s user 0.006s sys 0.038s
I20260809 15:52:47.548321 11011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50528 (request call id 2088) took 1360 ms. Trace:
W20260809 15:52:47.463135 14055 log.cc:924] Time spent T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.240s user 0.004s sys 0.003s
I20260809 15:52:47.558485 14047 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:47.473359 14000 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:47.370774 13857 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:47.626816 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 1.09006e+06us
W20260809 15:52:47.549126 14025 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.568s user 0.006s sys 0.027s
I20260809 15:52:47.624063 11642 rpcz_store.cc:276] 0809 15:52:45.026293 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.281736 (+1255443us) service_pool.cc:227] Handling call
0809 15:52:46.333840 (+ 52104us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.377454 (+ 43614us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:47.493226 (+1115772us) spinlock_profiling.cc:243] Waited 42.9 ms on lock 0x7b54006a63d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:47.493475 (+ 249us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":98697472}
I20260809 15:52:47.627576 11640 rpcz_store.cc:276] 0809 15:52:45.071191 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:45.222359 (+151168us) service_pool.cc:227] Handling call
0809 15:52:47.491351 (+2268992us) spinlock_profiling.cc:243] Waited 103 ms on lock 0x7b5400533754. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:47.491562 (+ 211us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":235994880}
I20260809 15:52:47.626051 10931 rpcz_store.cc:276] 0809 15:52:45.384443 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:45.384895 (+ 452us) service_pool.cc:227] Handling call
0809 15:52:45.385174 (+ 279us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:46.827536 (+1442362us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:47.512973 (+685437us) spinlock_profiling.cc:243] Waited 1.42 s on lock 0x7b540025cbd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:47.513212 (+ 239us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3272481152}
I20260809 15:52:47.570580 10844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2230) took 1458 ms. Trace:
W20260809 15:52:47.636266 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 96004.4us
I20260809 15:52:47.628721 11011 rpcz_store.cc:276] 0809 15:52:46.132852 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.161611 (+ 28759us) service_pool.cc:227] Handling call
0809 15:52:47.492587 (+1330976us) spinlock_profiling.cc:243] Waited 240 ms on lock 0x7b540019f0d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:47.492722 (+ 135us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":551837568}
I20260809 15:52:47.624289 11015 rpcz_store.cc:276] 0809 15:52:46.110203 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.110413 (+ 210us) service_pool.cc:227] Handling call
0809 15:52:46.119797 (+ 9384us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.879441 (+759644us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:47.498974 (+619533us) spinlock_profiling.cc:243] Waited 744 ms on lock 0x7b54000127d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:47.499207 (+ 233us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1711316736}
I20260809 15:52:47.624919 11641 rpcz_store.cc:276] 0809 15:52:43.993001 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:44.133078 (+140077us) service_pool.cc:227] Handling call
0809 15:52:44.133395 (+ 317us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:45.245042 (+1111647us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:47.508393 (+2263351us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b5400133b54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:47.508662 (+ 269us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2554837248}
W20260809 15:52:47.649571 14042 log.cc:924] Time spent T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.439s user 0.000s sys 0.002s
I20260809 15:52:47.649477 11606 leader_election.cc:302] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:47.641099 14000 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:47.315987 14019 consensus_queue.cc:237] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:47.616972 10837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2219) took 3544 ms. Trace:
I20260809 15:52:47.622916 10930 rpcz_store.cc:276] 0809 15:52:45.874616 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:45.915206 (+ 40590us) service_pool.cc:227] Handling call
0809 15:52:45.915559 (+ 353us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.143272 (+227713us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:47.517264 (+1373992us) spinlock_profiling.cc:243] Waited 71.4 ms on lock 0x7b540024d1d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:47.517486 (+ 222us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":164201344}
W20260809 15:52:47.569824 13882 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.991s user 0.024s sys 0.016s
W20260809 15:52:47.579216 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 244826us
W20260809 15:52:47.622880 10809 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.968s user 0.026s sys 0.074s
I20260809 15:52:47.410233 14048 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:47.539162 14029 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:47.538118 13913 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:47.555269 13820 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:47.552680 13628 raft_consensus.cc:2781] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election won for term 8
W20260809 15:52:47.716938 14069 log.cc:924] Time spent T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.125s user 0.000s sys 0.000s
I20260809 15:52:47.718928 14043 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:47.622839 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 60422.5us
I20260809 15:52:47.572525 14052 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:48.632027 10974 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.248s user 0.010s sys 0.107s
W20260809 15:52:48.678730 10974 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.364s user 0.012s sys 0.110s
I20260809 15:52:47.721139 10837 rpcz_store.cc:276] 0809 15:52:44.072510 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:44.072622 (+ 112us) service_pool.cc:227] Handling call
0809 15:52:44.072943 (+ 321us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:44.073138 (+ 195us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:44.073162 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:44.108774 (+ 35612us) log.cc:842] Serialized 26 byte log entry
0809 15:52:44.515206 (+406432us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:52:44.515243 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:44.515309 (+ 66us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:47.171442 (+2656133us) spinlock_profiling.cc:243] Waited 89.8 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:52:47.599778 (+428336us) raft_consensus.cc:1671] finished
0809 15:52:47.599806 (+ 28us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:47.616829 (+ 17023us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":207113344,"thread_start_us":344087,"threads_started":1,"wal-append.queue_time_us":386544,"wal-append.run_cpu_time_us":25712,"wal-append.run_wall_time_us":4083065}
W20260809 15:52:48.518946 10847 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.504s user 0.030s sys 0.058s
W20260809 15:52:48.681386 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 2.10997e+06us
W20260809 15:52:48.681928 10847 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.667s user 0.031s sys 0.063s
I20260809 15:52:47.790421 11640 raft_consensus.cc:1261] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
W20260809 15:52:48.474030 11014 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.458s user 0.022s sys 0.024s
W20260809 15:52:49.393693 11014 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 2.387s user 0.026s sys 0.025s
I20260809 15:52:47.627285 11017 rpcz_store.cc:276] 0809 15:52:46.102098 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.103074 (+ 976us) service_pool.cc:227] Handling call
0809 15:52:46.103376 (+ 302us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.167862 (+ 64486us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:46.167915 (+ 53us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:46.167967 (+ 52us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:46.168010 (+ 43us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:47.518386 (+1350376us) spinlock_profiling.cc:243] Waited 64.3 ms on lock 0x7b540002d854. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:47.518462 (+ 76us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:47.518617 (+ 155us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":147789568}
I20260809 15:52:48.683372 10847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2229) took 2571 ms. Trace:
I20260809 15:52:49.396413 11640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 901) took 3397 ms. Trace:
I20260809 15:52:49.396821 11640 rpcz_store.cc:276] 0809 15:52:45.999571 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:47.659602 (+1660031us) service_pool.cc:227] Handling call
0809 15:52:47.790049 (+130447us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:49.396141 (+1606092us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:49.396361 (+ 220us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:52:49.062294 10890 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.233s user 0.029s sys 0.041s
W20260809 15:52:49.038143 11010 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.848s user 0.018s sys 0.034s
W20260809 15:52:49.397821 10890 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.113s user 0.032s sys 0.047s
W20260809 15:52:49.398063 11010 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 2.208s user 0.021s sys 0.036s
W20260809 15:52:49.045131 11009 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.718s user 0.011s sys 0.007s
W20260809 15:52:49.398659 11009 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.071s user 0.011s sys 0.008s
W20260809 15:52:48.675230 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 1.02583e+06us
I20260809 15:52:47.980593 11639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 886) took 3835 ms. Trace:
I20260809 15:52:49.402371 11639 rpcz_store.cc:276] 0809 15:52:44.144904 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:44.683565 (+538661us) service_pool.cc:227] Handling call
0809 15:52:44.683870 (+ 305us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:44.887243 (+203373us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:47.980260 (+3093017us) spinlock_profiling.cc:243] Waited 201 ms on lock 0x7b540008a054. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:47.980500 (+ 240us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":462982912}
W20260809 15:52:48.564441 10810 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.250s user 0.041s sys 0.076s
W20260809 15:52:49.403908 10810 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.090s user 0.042s sys 0.089s
I20260809 15:52:47.786203 13906 raft_consensus.cc:3037] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
W20260809 15:52:47.662938 14061 log.cc:924] Time spent T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.169s user 0.000s sys 0.000s
I20260809 15:52:48.930195 11016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2078) took 4336 ms. Trace:
W20260809 15:52:48.083475 14031 consensus_meta.cc:287] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 1.895s user 0.012s sys 0.006s
I20260809 15:52:47.659204 14039 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:49.410863 10926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 769) took 3021 ms. Trace:
W20260809 15:52:49.411634 11015 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.156s user 0.000s sys 0.016s
W20260809 15:52:49.355638 14077 log.cc:924] Time spent T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.101s user 0.000s sys 0.001s
I20260809 15:52:49.395532 10847 rpcz_store.cc:276] 0809 15:52:46.111695 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.112500 (+ 805us) service_pool.cc:227] Handling call
0809 15:52:46.145648 (+ 33148us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.145893 (+ 245us) raft_consensus.cc:1520] Early marking committed up to index 6
0809 15:52:46.332772 (+186879us) log.cc:842] Serialized 14 byte log entry
0809 15:52:47.015058 (+682286us) log.cc:842] Serialized 14 byte log entry
0809 15:52:48.682586 (+1667528us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:48.682661 (+ 75us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:48.682732 (+ 71us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:48.683132 (+ 400us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:48.683301 (+ 169us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":91561,"prepare.queue_time_us":13757,"prepare.run_cpu_time_us":88,"prepare.run_wall_time_us":1207,"raft.queue_time_us":894061,"raft.run_cpu_time_us":452,"raft.run_wall_time_us":82700,"thread_start_us":2337129,"threads_started":2,"wal-append.queue_time_us":983462}
I20260809 15:52:48.813012 14000 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:47.663569 14062 log.cc:924] Time spent T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.310s user 0.000s sys 0.001s
W20260809 15:52:49.387478 10976 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.528s user 0.004s sys 0.008s
W20260809 15:52:49.465492 10976 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.650s user 0.006s sys 0.010s
W20260809 15:52:49.431825 11603 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.757s user 0.004s sys 0.009s
W20260809 15:52:49.465921 11603 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.791s user 0.005s sys 0.009s
I20260809 15:52:48.981333 14075 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:52:49.387986 10839 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 2.062s user 0.000s sys 0.014s
I20260809 15:52:49.302521 14048 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:49.466950 10839 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.141s user 0.000s sys 0.018s
I20260809 15:52:49.455679 10926 rpcz_store.cc:276] 0809 15:52:46.389437 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.389880 (+ 443us) service_pool.cc:227] Handling call
0809 15:52:46.390269 (+ 389us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:46.400932 (+ 10663us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:46.400976 (+ 44us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:46.404944 (+ 3968us) log.cc:842] Serialized 26 byte log entry
0809 15:52:47.225605 (+820661us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:47.225681 (+ 76us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:47.225837 (+ 156us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:48.106394 (+880557us) raft_consensus.cc:1671] finished
0809 15:52:48.106423 (+ 29us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:49.410786 (+1304363us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4047,"raft.queue_time_us":4154,"raft.run_cpu_time_us":246,"raft.run_wall_time_us":8038,"thread_start_us":700755,"threads_started":1,"wal-append.queue_time_us":1591639}
I20260809 15:52:49.467612 10839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 748) took 3067 ms. Trace:
I20260809 15:52:47.972325 14038 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:47.567772 13656 leader_election.cc:288] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:49.422523 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:47.627767 14028 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:49.148874 10892 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.298s user 0.023s sys 0.023s
W20260809 15:52:49.508152 10892 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.281s user 0.026s sys 0.030s
I20260809 15:52:48.814417 14029 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:47.945401 13867 consensus_meta.cc:287] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 1.538s user 0.008s sys 0.007s
I20260809 15:52:49.408221 13882 raft_consensus.cc:686] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
W20260809 15:52:48.474938 11632 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.712s user 0.025s sys 0.022s
W20260809 15:52:49.447270 13960 log.cc:924] Time spent T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 4.776s user 0.034s sys 0.057s
I20260809 15:52:47.721571 13881 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:52:48.518710 13594 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.779s user 0.028s sys 0.066s
I20260809 15:52:47.756410 13894 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:49.329591 10975 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 1.855s user 0.000s sys 0.013s
W20260809 15:52:49.519048 10975 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 2.045s user 0.000s sys 0.014s
W20260809 15:52:49.378875 10929 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.743s user 0.003s sys 0.010s
I20260809 15:52:48.930092 10809 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:47.659334 13982 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:47.583003 14054 raft_consensus.cc:2726] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260809 15:52:48.247140 14006 consensus_meta.cc:287] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 1.231s user 0.004s sys 0.009s
W20260809 15:52:48.308005 13925 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.307s user 0.026s sys 0.052s
W20260809 15:52:48.970363 13809 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 2.296s user 0.032s sys 0.068s
W20260809 15:52:49.519752 13594 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 2.780s user 0.038s sys 0.075s
W20260809 15:52:48.613870 13905 consensus_meta.cc:287] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 1.598s user 0.008s sys 0.007s
I20260809 15:52:49.444475 11016 rpcz_store.cc:276] 0809 15:52:44.480214 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:44.480730 (+ 516us) service_pool.cc:227] Handling call
0809 15:52:44.481101 (+ 371us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:44.493383 (+ 12282us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:44.493414 (+ 31us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:44.510830 (+ 17416us) log.cc:842] Serialized 26 byte log entry
0809 15:52:44.959600 (+448770us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:44.959638 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:44.959719 (+ 81us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:47.504349 (+2544630us) spinlock_profiling.cc:243] Waited 6.76 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:52:47.756908 (+252559us) raft_consensus.cc:1671] finished
0809 15:52:47.756932 (+ 24us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:48.816231 (+1059299us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":15169,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":21125,"spinlock_wait_cycles":15548288,"thread_start_us":375780,"threads_started":1,"wal-append.queue_time_us":593706,"wal-append.run_cpu_time_us":9844,"wal-append.run_wall_time_us":3673478}
I20260809 15:52:47.744082 13878 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:49.455303 11015 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.200s user 0.000s sys 0.016s
W20260809 15:52:49.411664 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 730853us
W20260809 15:52:48.998541 13473 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 2.492s user 0.045s sys 0.123s
W20260809 15:52:48.663147 13993 log.cc:924] Time spent T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 3.040s user 0.005s sys 0.032s
I20260809 15:52:48.462751 13806 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:47.744948 14005 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:47.926240 13837 consensus_meta.cc:287] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 1.317s user 0.011s sys 0.005s
I20260809 15:52:47.743713 13810 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:48.631223 14090 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:49.399060 13628 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 15:52:47.941758 14070 log.cc:924] Time spent T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.466s user 0.001s sys 0.001s
I20260809 15:52:49.511370 14075 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:49.462916 11642 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.669s user 0.004s sys 0.020s
W20260809 15:52:47.785722 14055 log.cc:924] Time spent T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.093s user 0.000s sys 0.000s
W20260809 15:52:47.742841 14063 log.cc:924] Time spent T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f: Append to log took a long time: real 0.148s user 0.000s sys 0.000s
W20260809 15:52:49.357406 14080 log.cc:924] Time spent T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.084s user 0.000s sys 0.000s
W20260809 15:52:48.254868 13762 consensus_meta.cc:287] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 1.358s user 0.015s sys 0.000s
I20260809 15:52:49.079882 13857 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:48.682890 13915 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:47.754978 13973 raft_consensus.cc:3032] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:49.398778 14053 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:52:49.468194 10978 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.652s user 0.004s sys 0.002s
W20260809 15:52:49.378226 14036 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 2.039s user 0.001s sys 0.013s
I20260809 15:52:49.468660 14000 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:49.472363 14025 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:49.472956 13464 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:49.473068 14091 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:49.474119 14026 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:49.481783 13656 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:47.629204 10920 rpcz_store.cc:276] 0809 15:52:44.906536 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:44.908483 (+ 1947us) service_pool.cc:227] Handling call
0809 15:52:44.911502 (+ 3019us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:45.154777 (+243275us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:47.509714 (+2354937us) spinlock_profiling.cc:243] Waited 221 ms on lock 0x7b5400189a54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:47.510038 (+ 324us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":508432768}
I20260809 15:52:49.483486 14102 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:49.483894 13860 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:49.484117 14094 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:52:49.489078 14078 log.cc:924] Time spent T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.095s user 0.000s sys 0.000s
I20260809 15:52:49.489415 14079 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:49.467947 10839 rpcz_store.cc:276] 0809 15:52:46.400670 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.534391 (+133721us) service_pool.cc:227] Handling call
0809 15:52:46.605883 (+ 71492us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.606143 (+ 260us) raft_consensus.cc:1520] Early marking committed up to index 6
0809 15:52:46.629438 (+ 23295us) log.cc:842] Serialized 14 byte log entry
0809 15:52:49.467262 (+2837824us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:49.467313 (+ 51us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:49.467370 (+ 57us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:49.467441 (+ 71us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:49.467563 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9255,"prepare.queue_time_us":23246,"prepare.run_cpu_time_us":32,"prepare.run_wall_time_us":33,"raft.queue_time_us":256589,"raft.run_cpu_time_us":162,"raft.run_wall_time_us":13144,"thread_start_us":2834005,"threads_started":2,"wal-append.queue_time_us":1675318,"wal-append.run_cpu_time_us":2211,"wal-append.run_wall_time_us":1003024}
I20260809 15:52:49.495980 13864 consensus_queue.cc:260] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:47.001019 11636 rpcz_store.cc:276] 0809 15:52:45.949878 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.956602 (+1006724us) service_pool.cc:227] Handling call
0809 15:52:46.986729 (+ 30127us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.996519 (+ 9790us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:46.996574 (+ 55us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:46.996616 (+ 42us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:46.996654 (+ 38us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:46.996709 (+ 55us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:46.996837 (+ 128us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9175,"raft.queue_time_us":11628}
I20260809 15:52:49.500694 14106 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:49.516889 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 115539us
I20260809 15:52:49.508726 14048 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:49.519511 10929 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.884s user 0.004s sys 0.011s
W20260809 15:52:49.519870 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 837431us
W20260809 15:52:49.514874 11632 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 2.752s user 0.027s sys 0.039s
I20260809 15:52:49.510689 14076 raft_consensus.cc:482] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:49.513916 13906 raft_consensus.cc:2726] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 15:52:49.466837 14039 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:48.011951 13890 consensus_meta.cc:287] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 1.256s user 0.015s sys 0.000s
I20260809 15:52:49.526291 11015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 624) took 3185 ms. Trace:
W20260809 15:52:49.523556 13925 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 2.525s user 0.029s sys 0.076s
W20260809 15:52:49.523453 13809 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 2.852s user 0.034s sys 0.075s
I20260809 15:52:49.520481 13881 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:47.744678 11634 raft_consensus.cc:2932] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Committing config change with OpId 4.7: config changed from index 6 to 7, VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) evicted. New config: { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:49.527143 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:49.578462 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 51972.4us
I20260809 15:52:49.576838 10841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45578 (request call id 1998) took 5401 ms. Trace:
I20260809 15:52:49.839488 10841 rpcz_store.cc:276] 0809 15:52:44.148936 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:44.198290 (+ 49354us) service_pool.cc:227] Handling call
0809 15:52:44.198586 (+ 296us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:44.200509 (+ 1923us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:44.200538 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:44.202646 (+ 2108us) log.cc:842] Serialized 26 byte log entry
0809 15:52:44.350705 (+148059us) raft_consensus.cc:1631] Marking committed up to 3
0809 15:52:44.350753 (+ 48us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:44.350837 (+ 84us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:49.446627 (+5095790us) spinlock_profiling.cc:243] Waited 3.7 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:52:49.549733 (+103106us) raft_consensus.cc:1671] finished
0809 15:52:49.549755 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:49.549938 (+ 183us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8504064,"thread_start_us":145617,"threads_started":1,"wal-append.queue_time_us":328006}
W20260809 15:52:49.838708 11642 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 2.045s user 0.004s sys 0.034s
I20260809 15:52:49.842839 11642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 900) took 3861 ms. Trace:
W20260809 15:52:49.836452 10978 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.021s user 0.004s sys 0.016s
I20260809 15:52:49.836467 11015 rpcz_store.cc:276] 0809 15:52:46.341297 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:47.700503 (+1359206us) service_pool.cc:227] Handling call
0809 15:52:47.793100 (+ 92597us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:47.925298 (+132198us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:47.925346 (+ 48us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:48.023913 (+ 98567us) log.cc:842] Serialized 26 byte log entry
0809 15:52:49.525891 (+1501978us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:49.525917 (+ 26us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:49.526017 (+ 100us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:49.526063 (+ 46us) raft_consensus.cc:1671] finished
0809 15:52:49.526073 (+ 10us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:49.526238 (+ 165us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1642381,"spinlock_wait_cycles":6078336,"thread_start_us":1270574,"threads_started":1,"wal-append.queue_time_us":1386258}
I20260809 15:52:49.844684 13992 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:49.850915 13864 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
W20260809 15:52:49.532986 13473 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 3.027s user 0.046s sys 0.131s
I20260809 15:52:49.851282 13992 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:49.521450 13894 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:49.852878 13907 raft_consensus.cc:3037] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:49.838542 14026 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:49.820791 14073 log.cc:924] Time spent T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.278s user 0.000s sys 0.001s
I20260809 15:52:49.524070 14006 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:49.837955 11606 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
W20260809 15:52:49.859771 14036 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 2.524s user 0.009s sys 0.017s
I20260809 15:52:49.839119 14091 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:49.672883 14029 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:49.855894 13464 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:49.980700 14006 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:49.983801 10891 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.514s user 0.011s sys 0.011s
W20260809 15:52:49.984148 10891 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.514s user 0.011s sys 0.011s
I20260809 15:52:49.527021 13905 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:49.987270 10893 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.510s user 0.004s sys 0.019s
I20260809 15:52:49.533192 13878 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:49.875226 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 50233.3us
W20260809 15:52:49.996106 10891 connection.cc:232] RPC call timeout handler was delayed by -1.1823056690045632s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
I20260809 15:52:49.644292 13930 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:49.837122 13628 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Advancing to term 8
W20260809 15:52:49.974707 14085 log.cc:924] Time spent T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.124s user 0.000s sys 0.000s
W20260809 15:52:49.985734 11014 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.591s user 0.007s sys 0.016s
I20260809 15:52:49.880885 14028 raft_consensus.cc:482] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:49.989398 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 151530us
I20260809 15:52:49.532469 13999 raft_consensus.cc:482] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:49.854445 13894 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:49.824156 13860 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:49.843266 11642 rpcz_store.cc:276] 0809 15:52:45.981881 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:47.659682 (+1677801us) service_pool.cc:227] Handling call
0809 15:52:47.791002 (+131320us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:47.791290 (+ 288us) raft_consensus.cc:1520] Early marking committed up to index 8
0809 15:52:47.791772 (+ 482us) log.cc:842] Serialized 14 byte log entry
0809 15:52:49.842402 (+2050630us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:49.842470 (+ 68us) raft_consensus.cc:1631] Marking committed up to 8
0809 15:52:49.842540 (+ 70us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:49.842626 (+ 86us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:49.842773 (+ 147us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":1071,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":35,"thread_start_us":2048347,"threads_started":1,"wal-append.queue_time_us":1688725}
W20260809 15:52:49.987676 10893 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.510s user 0.004s sys 0.019s
I20260809 15:52:49.881536 13836 raft_consensus.cc:3032] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:52:49.526065 14047 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:49.878865 13973 raft_consensus.cc:729] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
W20260809 15:52:50.006505 11014 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.612s user 0.008s sys 0.016s
I20260809 15:52:49.979082 13982 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:49.969226 13925 raft_consensus.cc:2726] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260809 15:52:50.019740 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 542155us
I20260809 15:52:49.947906 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:49.880007 14079 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:50.032130 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:50.032370 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:49.984871 14006 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:49.978323 14052 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:49.850402 13915 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:49.859728 14005 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:49.855767 13806 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:49.855173 13656 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:49.855603 14048 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:49.855774 13762 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:49.918102 13810 consensus_queue.cc:1046] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.145s
I20260809 15:52:49.645717 13882 consensus_queue.cc:237] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 3, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:50.140527 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 144116us
I20260809 15:52:50.211745 11632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 895) took 4932 ms. Trace:
I20260809 15:52:50.033876 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:50.103575 11014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 621) took 3936 ms. Trace:
I20260809 15:52:50.023864 13820 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:49.883220 14000 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:49.880102 14025 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:49.855038 13890 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:50.251358 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:50.053097 10978 connection.cc:232] RPC call timeout handler was delayed by -3.1824551370227709s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1905s for a response
I20260809 15:52:50.282040 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:50.370323 10978 connection.cc:232] RPC call timeout handler was delayed by -2.9583551370305941s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2154s for a response
I20260809 15:52:50.143692 11013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2322) took 5269 ms. Trace:
I20260809 15:52:49.995060 13913 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:49.995296 13837 raft_consensus.cc:482] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:49.880327 14102 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:50.034224 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:49.877391 14108 consensus_queue.cc:1035] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.223s
I20260809 15:52:50.329276 11015 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:49.884550 14093 raft_consensus.cc:482] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:49.884178 14090 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:49.986901 13906 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:50.018782 13299 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:49.995822 14054 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:50.158239 13809 consensus_queue.cc:1046] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.133s
I20260809 15:52:50.005985 13594 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:50.023826 14087 raft_consensus.cc:2781] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:50.384811 13656 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:50.338807 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 321327us
I20260809 15:52:50.022552 13836 raft_consensus.cc:729] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
W20260809 15:52:50.204653 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 186751us
W20260809 15:52:50.330307 14078 log.cc:924] Time spent T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.134s user 0.000s sys 0.000s
I20260809 15:52:50.346683 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:50.553725 13836 consensus_queue.cc:260] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:50.554710 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:50.555676 13836 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:49.867085 14076 raft_consensus.cc:504] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:50.556180 13890 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:50.558130 14076 leader_election.cc:288] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:49.880194 14094 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:50.346105 11014 rpcz_store.cc:276] 0809 15:52:46.087640 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.087761 (+ 121us) service_pool.cc:227] Handling call
0809 15:52:46.100571 (+ 12810us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.100834 (+ 263us) raft_consensus.cc:1520] Early marking committed up to index 6
0809 15:52:47.006838 (+906004us) log.cc:842] Serialized 14 byte log entry
0809 15:52:50.023481 (+3016643us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:50.023537 (+ 56us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:50.023580 (+ 43us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:50.023665 (+ 85us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:50.023775 (+ 110us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":1246188,"prepare.run_cpu_time_us":57,"prepare.run_wall_time_us":58,"raft.queue_time_us":119227,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":5451,"thread_start_us":3921680,"threads_started":3,"wal-append.queue_time_us":2175493}
I20260809 15:52:49.838814 14053 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:49.879834 14106 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:50.357986 13464 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:50.361996 11009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 623) took 4021 ms. Trace:
I20260809 15:52:50.631979 11009 rpcz_store.cc:276] 0809 15:52:46.340956 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:47.322031 (+981075us) service_pool.cc:227] Handling call
0809 15:52:47.322467 (+ 436us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:49.399051 (+2076584us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:49.399085 (+ 34us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:49.433437 (+ 34352us) log.cc:842] Serialized 26 byte log entry
0809 15:52:49.945979 (+512542us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:49.946034 (+ 55us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:49.946139 (+ 105us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:50.361624 (+415485us) raft_consensus.cc:1671] finished
0809 15:52:50.361656 (+ 32us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:50.361824 (+ 168us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":331839,"raft.run_cpu_time_us":293,"raft.run_wall_time_us":91940,"thread_start_us":2551131,"threads_started":2,"wal-append.queue_time_us":648924}
I20260809 15:52:50.557717 14000 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:50.363574 14091 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:50.018070 13930 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:50.132479 13999 raft_consensus.cc:504] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:50.383718 14124 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:50.039467 14028 raft_consensus.cc:504] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:50.037803 14047 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:50.341930 13982 consensus_queue.cc:1035] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.127s
I20260809 15:52:50.339711 14006 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:50.384826 14048 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:50.362843 14092 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:50.365651 13820 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:50.512013 13905 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:50.512303 13878 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:50.472776 14075 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:50.468650 14026 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:50.368964 13881 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:50.368790 14039 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:50.372581 14052 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:52:50.660672 13907 consensus_meta.cc:287] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.807s user 0.014s sys 0.000s
I20260809 15:52:50.633503 14054 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:50.636108 13906 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:51.130386 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:51.131861 11642 raft_consensus.cc:2454] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 1.
W20260809 15:52:51.020931 10811 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.666s user 0.001s sys 0.001s
W20260809 15:52:51.136248 10811 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.782s user 0.002s sys 0.001s
W20260809 15:52:51.136574 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 786347us
I20260809 15:52:51.137028 13915 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:50.342717 10809 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:51.023780 10810 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.952s user 0.000s sys 0.005s
I20260809 15:52:50.330219 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:51.161073 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:50.372928 10920 raft_consensus.cc:2379] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 7: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:51.161881 10921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54224 (request call id 1986) took 1119 ms. Trace:
I20260809 15:52:51.162369 10921 rpcz_store.cc:276] 0809 15:52:50.042609 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:50.099389 (+ 56780us) service_pool.cc:227] Handling call
0809 15:52:51.161640 (+1062251us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:51.162748 10920 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54224 (request call id 1979) took 1131 ms. Trace:
I20260809 15:52:51.163439 10920 rpcz_store.cc:276] 0809 15:52:50.031516 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:50.031714 (+ 198us) service_pool.cc:227] Handling call
0809 15:52:51.162517 (+1130803us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:51.163906 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:51.164915 10921 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
W20260809 15:52:51.165513 13836 consensus_meta.cc:287] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.609s user 0.008s sys 0.012s
W20260809 15:52:51.045799 10922 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.709s user 0.000s sys 0.045s
W20260809 15:52:51.166332 10922 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.830s user 0.000s sys 0.045s
W20260809 15:52:51.166707 10840 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2236) took 5017 ms (client timeout 1555 ms). Trace:
I20260809 15:52:51.046482 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:51.167196 10840 rpcz_store.cc:269] 0809 15:52:46.149186 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.150518 (+ 1332us) service_pool.cc:227] Handling call
0809 15:52:51.149068 (+4998550us) spinlock_profiling.cc:243] Waited 3.72 s on lock 0x7b5400059bd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:51.166441 (+ 17373us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8562164992}
I20260809 15:52:51.168722 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:50.335752 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:51.174098 14033 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:51.174901 10926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54224 (request call id 1988) took 1029 ms. Trace:
I20260809 15:52:51.175333 10926 rpcz_store.cc:276] 0809 15:52:50.145786 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:50.335410 (+189624us) service_pool.cc:227] Handling call
0809 15:52:51.174731 (+839321us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:52:51.047964 10929 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.223s user 0.008s sys 0.031s
I20260809 15:52:50.556208 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:50.492810 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:50.470618 10927 raft_consensus.cc:1226] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 7. Current term is 8. Ops: []
W20260809 15:52:50.375794 10978 connection.cc:232] RPC call timeout handler was delayed by -2.1966079979902133s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2136s for a response
W20260809 15:52:51.203724 10978 connection.cc:232] RPC call timeout handler was delayed by -2.1392226860043593s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16990000000000002s for a response
W20260809 15:52:51.204201 10978 connection.cc:232] RPC call timeout handler was delayed by -2.1392226860043593s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16990000000000002s for a response
W20260809 15:52:51.204517 10978 connection.cc:232] RPC call timeout handler was delayed by -2.0825226859888062s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17620000000000002s for a response
W20260809 15:52:51.204830 10978 connection.cc:232] RPC call timeout handler was delayed by -2.0825226859888062s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17620000000000002s for a response
W20260809 15:52:51.205123 10978 connection.cc:232] RPC call timeout handler was delayed by -2.0816226860042661s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1763s for a response
W20260809 15:52:51.205425 10978 connection.cc:232] RPC call timeout handler was delayed by -2.0816226860042661s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1763s for a response
W20260809 15:52:51.205711 10978 connection.cc:232] RPC call timeout handler was delayed by -1.9502226859913208s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19090000000000001s for a response
I20260809 15:52:51.050138 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:51.206771 11009 raft_consensus.cc:2379] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 2: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
W20260809 15:52:51.206945 13864 consensus_meta.cc:287] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 1.355s user 0.012s sys 0.007s
W20260809 15:52:51.207317 11009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39978 (request call id 629) took 2293 ms (client timeout 2213 ms). Trace:
W20260809 15:52:51.207731 11009 rpcz_store.cc:269] 0809 15:52:48.913925 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.049129 (+2135204us) service_pool.cc:227] Handling call
0809 15:52:51.207188 (+158059us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:51.208961 11012 raft_consensus.cc:1261] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260809 15:52:50.514302 11018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2086) took 4385 ms. Trace:
I20260809 15:52:51.209987 10931 raft_consensus.cc:2454] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:52:51.209918 11018 rpcz_store.cc:276] 0809 15:52:46.128841 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.564729 (+435888us) service_pool.cc:227] Handling call
0809 15:52:46.625384 (+ 60655us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.701543 (+ 76159us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:46.701573 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:46.701609 (+ 36us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:46.701636 (+ 27us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:50.513946 (+3812310us) spinlock_profiling.cc:243] Waited 76 ms on lock 0x7b540040a954. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:50.514022 (+ 76us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:50.514221 (+ 199us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":174750592}
I20260809 15:52:51.211261 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:51.211717 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:51.212739 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:50.594594 11015 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39978 (request call id 628) took 1913 ms (client timeout 2229 ms). Trace:
I20260809 15:52:51.213632 10931 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
W20260809 15:52:51.214062 11015 rpcz_store.cc:269] 0809 15:52:48.681268 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:49.866205 (+1184937us) service_pool.cc:227] Handling call
0809 15:52:50.594495 (+728290us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:51.214478 14051 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:51.215425 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:51.215133 14051 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:51.053606 11016 raft_consensus.cc:2454] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:51.216275 11015 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:51.216944 11015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50542 (request call id 2341) took 1238 ms. Trace:
W20260809 15:52:51.217499 11016 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39978 (request call id 632) took 2257 ms (client timeout 1719 ms). Trace:
W20260809 15:52:51.218111 11016 rpcz_store.cc:269] 0809 15:52:48.960104 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:49.526822 (+566718us) service_pool.cc:227] Handling call
0809 15:52:51.217100 (+1690278us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b54001a59d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:51.217300 (+ 200us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2825277184}
I20260809 15:52:51.218675 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:50.387262 11017 raft_consensus.cc:2379] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 10: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:51.219573 10930 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:51.220605 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:51.221093 11017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39978 (request call id 630) took 2306 ms (client timeout 1729 ms). Trace:
I20260809 15:52:51.220177 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
W20260809 15:52:51.222251 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 879425us
I20260809 15:52:51.223357 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:51.224994 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:50.345880 11632 rpcz_store.cc:276] 0809 15:52:45.280345 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.753066 (+1472721us) service_pool.cc:227] Handling call
0809 15:52:46.753559 (+ 493us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.753933 (+ 374us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 15:52:49.859545 (+3105612us) log.cc:842] Serialized 14 byte log entry
0809 15:52:50.024334 (+164789us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:50.099510 (+ 75176us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:50.129492 (+ 29982us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:50.143531 (+ 14039us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:50.158844 (+ 15313us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":26034,"prepare.queue_time_us":562629,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":3124,"raft.run_cpu_time_us":141,"raft.run_wall_time_us":76395,"thread_start_us":3153247,"threads_started":2,"wal-append.queue_time_us":412491}
I20260809 15:52:50.384936 11633 raft_consensus.cc:1226] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:51.226552 14133 raft_consensus.cc:2781] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:51.227099 14133 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:51.068004 11637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 890) took 6356 ms. Trace:
I20260809 15:52:51.068988 11635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41478 (request call id 979) took 7176 ms. Trace:
W20260809 15:52:50.646236 11604 connection.cc:232] RPC call timeout handler was delayed by -1.5492770619457588s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1527s for a response
W20260809 15:52:50.647064 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 93676us
I20260809 15:52:50.635803 13299 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:50.624341 13594 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:51.086990 13762 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:50.345507 13810 consensus_queue.cc:1225] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer 20730d75ad954a82b126148f8152a9f3 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260809 15:52:50.623433 13806 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:50.388562 13837 raft_consensus.cc:504] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:51.098932 10843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2238) took 4921 ms. Trace:
I20260809 15:52:50.658849 13860 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:51.116425 11641 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41488 (request call id 902) took 3360 ms (client timeout 2661 ms). Trace:
I20260809 15:52:51.123142 13822 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:51.132472 13906 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:50.023685 10842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 743) took 4112 ms. Trace:
I20260809 15:52:51.132488 13900 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:50.388387 13913 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:51.133028 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:51.135015 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:51.139950 13930 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:51.138693 13905 consensus_queue.cc:1035] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 15:52:50.658351 13987 consensus_queue.cc:1035] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.283s
W20260809 15:52:50.922200 13989 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 5.835s user 0.056s sys 0.104s
I20260809 15:52:50.920642 14005 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:50.984892 14006 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:50.948401 14025 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:50.979892 14028 leader_election.cc:288] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:50.920949 14052 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:50.983446 14054 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:50.973469 14079 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:50.649525 14087 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:50.976588 14090 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:50.905246 14092 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:50.672173 14093 raft_consensus.cc:504] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:50.968961 14094 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:50.939577 14102 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:50.978109 14106 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:50.506203 14123 consensus_queue.cc:1046] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.344s
I20260809 15:52:50.761775 14124 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:50.971102 14132 log.cc:924] Time spent T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.051s user 0.000s sys 0.000s
W20260809 15:52:51.155176 10810 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 1.083s user 0.000s sys 0.005s
I20260809 15:52:51.165062 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:50.368856 10923 raft_consensus.cc:2454] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:51.173678 10924 raft_consensus.cc:2454] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:51.178608 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:51.188642 10929 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.363s user 0.008s sys 0.031s
I20260809 15:52:51.175359 14033 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:51.196805 14000 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:51.191109 13915 consensus_queue.cc:1035] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.006s
I20260809 15:52:50.378058 11010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2328) took 5042 ms. Trace:
W20260809 15:52:51.215577 11634 consensus_meta.cc:287] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 1.391s user 0.009s sys 0.016s
I20260809 15:52:50.370818 11013 rpcz_store.cc:276] 0809 15:52:44.873861 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:44.874154 (+ 293us) service_pool.cc:227] Handling call
0809 15:52:44.874453 (+ 299us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:44.880218 (+ 5765us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:44.880245 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:44.880662 (+ 417us) log.cc:842] Serialized 26 byte log entry
0809 15:52:44.988667 (+108005us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:44.988730 (+ 63us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:44.988838 (+ 108us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:48.118110 (+3129272us) spinlock_profiling.cc:243] Waited 10.6 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:52:50.143311 (+2025201us) raft_consensus.cc:1671] finished
0809 15:52:50.143341 (+ 30us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:50.143617 (+ 276us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5412,"raft.queue_time_us":10747,"raft.run_cpu_time_us":79,"raft.run_wall_time_us":79,"spinlock_wait_cycles":24325376,"thread_start_us":100432,"threads_started":1,"wal-append.queue_time_us":170899}
I20260809 15:52:50.646620 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:51.227450 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:51.420135 14087 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:51.212522 11018 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:51.212580 11012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2332) took 5553 ms. Trace:
I20260809 15:52:51.217249 11015 rpcz_store.cc:276] 0809 15:52:49.978678 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.215067 (+1236389us) service_pool.cc:227] Handling call
0809 15:52:51.216800 (+ 1733us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:51.284420 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:51.206008 10978 connection.cc:232] RPC call timeout handler was delayed by -1.9502226859913208s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19090000000000001s for a response
W20260809 15:52:51.286760 10893 connection.cc:232] RPC call timeout handler was delayed by -3.4190966510213912s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.3227s for a response
W20260809 15:52:51.427392 10893 connection.cc:232] RPC call timeout handler was delayed by -3.34867152600782s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
I20260809 15:52:51.332422 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:51.422632 11641 rpcz_store.cc:269] 0809 15:52:47.756186 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.115795 (+3359609us) service_pool.cc:214] Skipping call since client already timed out
0809 15:52:51.116097 (+ 302us) inbound_call.cc:173] Queueing failure response
Metrics: {}
W20260809 15:52:51.221585 11017 rpcz_store.cc:269] 0809 15:52:48.914270 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:49.422030 (+507760us) service_pool.cc:227] Handling call
0809 15:52:51.220690 (+1798660us) spinlock_profiling.cc:243] Waited 911 ms on lock 0x7b54002c5454. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:51.220900 (+ 210us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2096310016}
I20260809 15:52:51.219584 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:51.422334 11018 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50542 (request call id 2347) took 1441 ms (client timeout 1791 ms). Trace:
W20260809 15:52:51.432118 10978 connection.cc:232] RPC call timeout handler was delayed by -1.6559226859826595s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22360000000000002s for a response
W20260809 15:52:51.432498 11018 rpcz_store.cc:269] 0809 15:52:49.980930 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.211372 (+1230442us) service_pool.cc:227] Handling call
0809 15:52:51.422226 (+210854us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:51.424474 11012 rpcz_store.cc:276] 0809 15:52:45.659207 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:45.659332 (+ 125us) service_pool.cc:227] Handling call
0809 15:52:45.659713 (+ 381us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:51.209565 (+5549852us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:51.212145 (+ 2580us) spinlock_profiling.cc:243] Waited 5.55 s on lock 0x7b5400180454. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:51.212469 (+ 324us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":12760403840}
I20260809 15:52:51.209308 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:51.227526 14133 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:51.424134 10843 rpcz_store.cc:276] 0809 15:52:46.177058 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.177206 (+ 148us) service_pool.cc:227] Handling call
0809 15:52:46.177485 (+ 279us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.390525 (+213040us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:51.098142 (+4707617us) spinlock_profiling.cc:243] Waited 209 ms on lock 0x7b5400005d54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:51.098591 (+ 449us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":481637632}
I20260809 15:52:51.429898 10924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 2036) took 1091 ms. Trace:
I20260809 15:52:51.439414 11016 raft_consensus.cc:2454] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:51.388403 10922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1984) took 1347 ms. Trace:
I20260809 15:52:51.288911 11633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 894) took 6036 ms. Trace:
W20260809 15:52:51.207736 13628 consensus_meta.cc:287] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 1.154s user 0.018s sys 0.003s
I20260809 15:52:51.424500 11637 rpcz_store.cc:276] 0809 15:52:44.710983 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:45.859589 (+1148606us) service_pool.cc:227] Handling call
0809 15:52:45.859958 (+ 369us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.702015 (+842057us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:51.067091 (+4365076us) spinlock_profiling.cc:243] Waited 839 ms on lock 0x7b5400089654. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:51.067634 (+ 543us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1929931520}
W20260809 15:52:51.440347 10978 connection.cc:232] RPC call timeout handler was delayed by -1.6559226859826595s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22360000000000002s for a response
I20260809 15:52:51.475196 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:51.423161 11635 rpcz_store.cc:276] 0809 15:52:43.891869 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:44.111033 (+219164us) service_pool.cc:227] Handling call
0809 15:52:45.813415 (+1702382us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:45.865932 (+ 52517us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:51.066419 (+5200487us) spinlock_profiling.cc:243] Waited 51.8 ms on lock 0x7b540066fdd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:51.068410 (+ 1991us) spinlock_profiling.cc:243] Waited 1.7 s on lock 0x7b540066fdd0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:51.068699 (+ 289us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4033486976}
W20260809 15:52:51.472812 10722 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.GetConsensusState -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 99160.7us
W20260809 15:52:51.490660 10978 connection.cc:232] RPC call timeout handler was delayed by -1.0141516249859706s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.4342s for a response
I20260809 15:52:51.462453 10926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 2043) took 1092 ms. Trace:
W20260809 15:52:51.475567 10893 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.84.1:46615 timed out after 1.629s (SENT) [suppressed 8 similar messages]
I20260809 15:52:51.474095 11640 raft_consensus.cc:2454] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
W20260809 15:52:51.472188 10891 leader_election.cc:334] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): Timed out: RequestConsensusVote RPC to 127.9.84.3:39223 timed out after 1.629s (SENT)
I20260809 15:52:51.473635 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:51.481587 11636 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:51.463035 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54224 (request call id 1991) took 1297 ms. Trace:
I20260809 15:52:51.492837 13822 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:51.431291 13989 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 6.344s user 0.060s sys 0.104s
I20260809 15:52:51.494280 10924 rpcz_store.cc:276] 0809 15:52:50.331535 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.168244 (+836709us) service_pool.cc:227] Handling call
0809 15:52:51.423291 (+255047us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:51.494585 11016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50542 (request call id 2346) took 1512 ms. Trace:
I20260809 15:52:51.552995 11016 rpcz_store.cc:276] 0809 15:52:49.980395 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.219223 (+1238828us) service_pool.cc:227] Handling call
0809 15:52:51.492324 (+273101us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:52:51.533970 11009 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50542 (request call id 2349) took 1552 ms (client timeout 1629 ms). Trace:
W20260809 15:52:51.553798 11009 rpcz_store.cc:269] 0809 15:52:49.981624 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.208929 (+1227305us) service_pool.cc:227] Handling call
0809 15:52:51.533913 (+324984us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:51.514813 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:51.497978 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:51.496206 11013 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:51.492465 10926 rpcz_store.cc:276] 0809 15:52:50.355984 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.178150 (+822166us) service_pool.cc:227] Handling call
0809 15:52:51.448156 (+270006us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:52:51.496986 10978 connection.cc:232] RPC call timeout handler was delayed by -1.0141516249859706s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.4342s for a response
I20260809 15:52:51.471263 13806 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:51.571352 13837 leader_election.cc:288] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:51.571985 13822 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:51.578683 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:51.578636 14090 consensus_queue.cc:1046] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:51.580621 13819 raft_consensus.cc:3032] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:52:51.581012 13819 raft_consensus.cc:729] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:47.633898 10844 rpcz_store.cc:276] 0809 15:52:46.112062 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.112188 (+ 126us) service_pool.cc:227] Handling call
0809 15:52:46.157424 (+ 45236us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:46.157651 (+ 227us) raft_consensus.cc:1520] Early marking committed up to index 6
0809 15:52:46.192268 (+ 34617us) log.cc:842] Serialized 14 byte log entry
0809 15:52:47.570129 (+1377861us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:47.570183 (+ 54us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:47.570261 (+ 78us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:47.570357 (+ 96us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:47.570498 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":26668,"prepare.queue_time_us":34360,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":76,"raft.queue_time_us":83293,"thread_start_us":1350476,"threads_started":1,"wal-append.queue_time_us":1141462}
I20260809 15:52:51.581581 13913 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:51.582538 13819 consensus_queue.cc:260] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:51.584597 13819 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:51.584906 11011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 625) took 5220 ms. Trace:
I20260809 15:52:51.586395 14075 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:51.585373 11011 rpcz_store.cc:276] 0809 15:52:46.364365 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:47.764579 (+1400214us) service_pool.cc:227] Handling call
0809 15:52:47.788358 (+ 23779us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:47.788778 (+ 420us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 15:52:47.916081 (+127303us) log.cc:842] Serialized 14 byte log entry
0809 15:52:51.083453 (+3167372us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b1800372be0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f79665ad345 00007f796f3e4a39 00007f79665ca8bb 00007f79665c8cbf 00007f79665cae22 00007f79691745bb 00007f796917a063 00007f796917a57b 00007f79694fd88a
0809 15:52:51.448515 (+365062us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:51.497668 (+ 49153us) log.cc:842] Serialized 26 byte log entry
0809 15:52:51.518117 (+ 20449us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:51.518176 (+ 59us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:51.518299 (+ 123us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:51.522238 (+ 3939us) raft_consensus.cc:1671] finished
0809 15:52:51.566688 (+ 44450us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:51.584690 (+ 18002us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":126955,"prepare.run_cpu_time_us":116,"prepare.run_wall_time_us":25795,"raft.queue_time_us":1640270,"raft.run_cpu_time_us":625,"raft.run_wall_time_us":201084,"spinlock_wait_cycles":23974656,"thread_start_us":332162,"threads_started":2,"wal-append.queue_time_us":3368501}
I20260809 15:52:51.587189 14075 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:51.588018 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:51.589221 11014 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:51.589497 14075 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:51.590348 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:51.590595 10929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1972) took 6233 ms. Trace:
I20260809 15:52:51.590950 10929 rpcz_store.cc:276] 0809 15:52:45.334041 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:45.334929 (+ 888us) service_pool.cc:227] Handling call
0809 15:52:45.381038 (+ 46109us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:45.381357 (+ 319us) raft_consensus.cc:1520] Early marking committed up to index 3
0809 15:52:46.285903 (+904546us) log.cc:842] Serialized 14 byte log entry
0809 15:52:47.634917 (+1349014us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:47.635459 (+ 542us) log.cc:842] Serialized 26 byte log entry
0809 15:52:47.635618 (+ 159us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:52:49.824277 (+2188659us) log.cc:842] Serialized 14 byte log entry
0809 15:52:51.474303 (+1650026us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:51.493978 (+ 19675us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:51.494022 (+ 44us) raft_consensus.cc:1671] finished
0809 15:52:51.494033 (+ 11us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:51.566891 (+ 72858us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1694651,"prepare.queue_time_us":2229705,"prepare.run_cpu_time_us":92,"prepare.run_wall_time_us":1637,"raft.queue_time_us":162087,"raft.run_cpu_time_us":500,"raft.run_wall_time_us":118183,"thread_start_us":5775066,"threads_started":4,"wal-append.queue_time_us":1927936,"wal-append.run_cpu_time_us":31250,"wal-append.run_wall_time_us":2492362}
I20260809 15:52:51.594591 13838 leader_election.cc:288] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:51.596338 14124 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:51.597659 10811 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:51.518553 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:51.515254 10930 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:51.441519 14123 raft_consensus.cc:3032] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:51.498134 11642 raft_consensus.cc:2379] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 6: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:51.601336 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:51.513914 10931 rpcz_store.cc:276] 0809 15:52:50.166036 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.226208 (+1060172us) service_pool.cc:227] Handling call
0809 15:52:51.462960 (+236752us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:51.574664 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:51.575920 14028 consensus_queue.cc:1046] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:51.577131 13999 leader_election.cc:288] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:51.512670 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:51.521153 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:51.593739 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:51.517112 10923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54224 (request call id 1985) took 1456 ms. Trace:
I20260809 15:52:51.598268 13958 raft_consensus.cc:3037] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:51.444099 14093 leader_election.cc:288] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:51.569350 13806 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:51.588637 11011 raft_consensus.cc:2454] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:51.573865 14047 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:51.616575 11604 connection.cc:232] RPC call timeout handler was delayed by -1.3764770619454794s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17190000000000003s for a response
I20260809 15:52:51.625062 11639 raft_consensus.cc:2379] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 4: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:51.516361 14092 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:51.460271 14094 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:51.620045 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:51.620718 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:51.498651 10920 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 2042) took 1150 ms. Trace:
I20260809 15:52:51.473220 13299 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:51.619231 10923 rpcz_store.cc:276] 0809 15:52:50.041512 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:50.042267 (+ 755us) service_pool.cc:227] Handling call
0809 15:52:51.494129 (+1451862us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:51.630188 11009 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
W20260809 15:52:51.631374 11604 connection.cc:232] RPC call timeout handler was delayed by -1.3674770619836636s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17290000000000003s for a response
I20260809 15:52:51.519958 13959 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:51.445874 13907 consensus_queue.cc:1046] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.171s
I20260809 15:52:51.633888 10926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54224 (request call id 1990) took 1476 ms. Trace:
I20260809 15:52:51.634224 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:51.638360 10926 rpcz_store.cc:276] 0809 15:52:50.157548 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.574223 (+1416675us) service_pool.cc:227] Handling call
0809 15:52:51.633832 (+ 59609us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:51.519652 14052 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:51.614948 14123 raft_consensus.cc:729] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:51.640329 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:51.642998 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:51.637857 11018 raft_consensus.cc:2454] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:51.645434 11018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50542 (request call id 2351) took 1299 ms. Trace:
I20260809 15:52:51.645849 11018 rpcz_store.cc:276] 0809 15:52:50.346964 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.459499 (+1112535us) service_pool.cc:227] Handling call
0809 15:52:51.645361 (+185862us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:52:51.647104 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 183749us
I20260809 15:52:51.491878 10922 rpcz_store.cc:276] 0809 15:52:50.041190 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:50.041310 (+ 120us) service_pool.cc:227] Handling call
0809 15:52:50.220625 (+179315us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:50.220861 (+ 236us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 15:52:51.166923 (+946062us) log.cc:842] Serialized 14 byte log entry
0809 15:52:51.331830 (+164907us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:51.385056 (+ 53226us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:51.385173 (+ 117us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:51.388128 (+ 2955us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:51.388310 (+ 182us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":69681,"prepare.queue_time_us":616347,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":49,"raft.queue_time_us":133972,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":27129,"thread_start_us":857395,"threads_started":2,"wal-append.queue_time_us":266781}
I20260809 15:52:51.456156 11010 rpcz_store.cc:276] 0809 15:52:45.335672 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:45.337291 (+ 1619us) service_pool.cc:227] Handling call
0809 15:52:45.379593 (+ 42302us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:47.012197 (+1632604us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:47.012243 (+ 46us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:47.015597 (+ 3354us) log.cc:842] Serialized 26 byte log entry
0809 15:52:49.398624 (+2383027us) raft_consensus.cc:1631] Marking committed up to 3
0809 15:52:49.398663 (+ 39us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:49.398874 (+ 211us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:50.377831 (+978957us) raft_consensus.cc:1671] finished
0809 15:52:50.377850 (+ 19us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:50.378000 (+ 150us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":9139,"raft.queue_time_us":483228,"raft.run_cpu_time_us":173,"raft.run_wall_time_us":173,"thread_start_us":3822747,"threads_started":2,"wal-append.queue_time_us":2338284,"wal-append.run_cpu_time_us":14584,"wal-append.run_wall_time_us":1986937}
I20260809 15:52:51.523108 13628 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:51.471832 14000 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:51.651186 11012 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:51.651473 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:51.622601 14124 raft_consensus.cc:2726] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:51.650312 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:51.654388 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:51.491775 11633 rpcz_store.cc:276] 0809 15:52:45.252095 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:46.750332 (+1498237us) service_pool.cc:227] Handling call
0809 15:52:46.750990 (+ 658us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:51.226645 (+4475655us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:51.227638 (+ 993us) spinlock_profiling.cc:243] Waited 3.61 s on lock 0x7b5400590bd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:51.288712 (+ 61074us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8312035072}
I20260809 15:52:51.679351 11013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50542 (request call id 2352) took 1306 ms. Trace:
I20260809 15:52:51.679787 11013 rpcz_store.cc:276] 0809 15:52:50.372652 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.627978 (+1255326us) service_pool.cc:227] Handling call
0809 15:52:51.679283 (+ 51305us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:51.680423 10923 raft_consensus.cc:1226] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260809 15:52:51.680789 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:51.681280 10923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54236 (request call id 2038) took 1349 ms. Trace:
I20260809 15:52:51.682335 14030 raft_consensus.cc:2726] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:52:51.683302 11009 raft_consensus.cc:2454] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:51.685091 11009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 627) took 5166 ms. Trace:
I20260809 15:52:51.685405 11009 rpcz_store.cc:276] 0809 15:52:46.518598 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.684050 (+5165452us) service_pool.cc:227] Handling call
0809 15:52:51.684449 (+ 399us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:51.684759 (+ 310us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:51.684786 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:51.684845 (+ 59us) raft_consensus.cc:1631] Marking committed up to 7
0809 15:52:51.684875 (+ 30us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:51.684924 (+ 49us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:51.685020 (+ 96us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:51.677243 11017 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:51.658694 13959 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:51.681949 11013 raft_consensus.cc:2396] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 15:52:51.660138 13594 leader_election.cc:288] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:51.662444 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 64982.7us
I20260809 15:52:51.656236 14106 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:51.736383 14123 consensus_queue.cc:260] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:51.655622 10929 raft_consensus.cc:2454] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:51.681826 10923 rpcz_store.cc:276] 0809 15:52:50.332046 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.644758 (+1312712us) service_pool.cc:227] Handling call
0809 15:52:51.679924 (+ 35166us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:51.680964 (+ 1040us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:51.681200 (+ 236us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:51.707083 10926 raft_consensus.cc:1261] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260809 15:52:52.180234 10926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54236 (request call id 2032) took 1849 ms. Trace:
I20260809 15:52:52.180675 10926 rpcz_store.cc:276] 0809 15:52:50.330393 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.647815 (+1317422us) service_pool.cc:227] Handling call
0809 15:52:51.706576 (+ 58761us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:52.179869 (+473293us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:52.180152 (+ 283us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.034583 10811 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:51.645102 10921 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:51.675949 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1992) took 1506 ms. Trace:
I20260809 15:52:52.198607 14049 raft_consensus.cc:2726] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:51.759752 14133 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:52.198791 10931 rpcz_store.cc:276] 0809 15:52:50.166396 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.638787 (+1472391us) service_pool.cc:227] Handling call
0809 15:52:51.661011 (+ 22224us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:51.661727 (+ 716us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 15:52:51.661776 (+ 49us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:51.661881 (+ 105us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:51.661934 (+ 53us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:51.662020 (+ 86us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:51.673102 (+ 11082us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":653,"raft.run_cpu_time_us":145,"raft.run_wall_time_us":146}
I20260809 15:52:51.718781 10891 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:52.201416 14133 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:52.202045 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 483381us
I20260809 15:52:52.202960 10847 raft_consensus.cc:1226] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:52.204605 13982 raft_consensus.cc:2781] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:52.205008 13982 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:52.205348 13982 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
W20260809 15:52:52.197883 10921 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 2041) took 1852 ms (client timeout 1818 ms). Trace:
W20260809 15:52:52.206228 10921 rpcz_store.cc:269] 0809 15:52:50.345581 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.163544 (+817963us) service_pool.cc:227] Handling call
0809 15:52:52.197772 (+1034228us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:51.709885 11009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 631) took 2795 ms. Trace:
I20260809 15:52:52.211304 11009 rpcz_store.cc:276] 0809 15:52:48.914596 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.708808 (+2794212us) service_pool.cc:227] Handling call
0809 15:52:51.709232 (+ 424us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:51.709555 (+ 323us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:51.709578 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:51.709615 (+ 37us) raft_consensus.cc:1631] Marking committed up to 8
0809 15:52:51.709649 (+ 34us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:51.709717 (+ 68us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:51.709814 (+ 97us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.212565 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.213487 11009 raft_consensus.cc:2396] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 7 because replica has last-logged OpId of term: 6 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260809 15:52:52.213572 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:51.712033 11016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50542 (request call id 2350) took 1612 ms. Trace:
I20260809 15:52:52.243228 11016 rpcz_store.cc:276] 0809 15:52:50.099585 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.600013 (+1500428us) service_pool.cc:227] Handling call
0809 15:52:51.711967 (+111954us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.244392 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:52.245028 11016 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:51.659636 11015 raft_consensus.cc:2454] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
W20260809 15:52:52.246995 11015 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50542 (request call id 2348) took 2265 ms (client timeout 2192 ms). Trace:
W20260809 15:52:52.247375 11015 rpcz_store.cc:269] 0809 15:52:49.981322 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.434350 (+1453028us) service_pool.cc:227] Handling call
0809 15:52:52.246926 (+812576us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.248251 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.249161 11015 raft_consensus.cc:2396] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260809 15:52:51.691157 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:52.251431 11011 raft_consensus.cc:3037] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:51.710326 11013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50542 (request call id 2353) took 1208 ms. Trace:
I20260809 15:52:52.254818 11013 rpcz_store.cc:276] 0809 15:52:50.505351 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.680468 (+1175117us) service_pool.cc:227] Handling call
0809 15:52:51.710269 (+ 29801us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.255820 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.256446 11013 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:52.257311 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:51.731068 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.259019 11012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39978 (request call id 636) took 1700 ms. Trace:
I20260809 15:52:52.259367 11012 rpcz_store.cc:276] 0809 15:52:50.558957 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.689582 (+1130625us) service_pool.cc:227] Handling call
0809 15:52:52.258961 (+569379us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.260319 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.261000 11012 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 10.
I20260809 15:52:51.691888 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.263350 11018 raft_consensus.cc:2396] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 8 because replica has last-logged OpId of term: 5 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 4.
I20260809 15:52:52.263813 11018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50542 (request call id 2354) took 1638 ms. Trace:
I20260809 15:52:52.264123 11018 rpcz_store.cc:276] 0809 15:52:50.625757 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.654761 (+1029004us) service_pool.cc:227] Handling call
0809 15:52:52.263761 (+609000us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.265007 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:51.687598 11017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50542 (request call id 2345) took 1707 ms. Trace:
W20260809 15:52:51.732885 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 45913.9us
I20260809 15:52:52.279568 10998 tablet_service.cc:1514] Processing DeleteTablet for tablet 5e8d12b5ef9e415cbcb25e62ab228546 with delete_type TABLET_DATA_TOMBSTONED (TS d74944fa9fa14ec78b0640ea709ff3aa not found in new config with opid_index 7) from {username='slave'} at 127.0.0.1:50526
I20260809 15:52:51.699703 11632 raft_consensus.cc:2379] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 8: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:51.772378 11634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 679) took 7403 ms. Trace:
I20260809 15:52:51.707329 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:51.708236 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.300390 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.299783 10736 catalog_manager.cc:5495] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 2 to 4, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 4 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } }
I20260809 15:52:52.301222 11637 raft_consensus.cc:2379] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 6: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:51.708287 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.303062 11639 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:51.738032 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:51.710005 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.306059 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:51.700130 13806 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:51.785880 13656 raft_consensus.cc:3032] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:51.734429 13893 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:51.720410 13999 consensus_queue.cc:1035] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 15:52:52.315478 13999 consensus_queue.cc:1225] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer d74944fa9fa14ec78b0640ea709ff3aa log is divergent from this leader: its last log entry 6.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260809 15:52:51.750062 14000 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:51.703826 14051 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:51.714902 14094 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:52.321682 14094 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:51.964233 10922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54236 (request call id 2040) took 1621 ms. Trace:
I20260809 15:52:52.322387 10922 rpcz_store.cc:276] 0809 15:52:50.343172 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.656134 (+1312962us) service_pool.cc:227] Handling call
0809 15:52:51.656809 (+ 675us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:51.657496 (+ 687us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:51.657540 (+ 44us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:51.658263 (+ 723us) log.cc:842] Serialized 26 byte log entry
0809 15:52:51.712038 (+ 53775us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:51.712081 (+ 43us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:51.712167 (+ 86us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:51.963950 (+251783us) raft_consensus.cc:1671] finished
0809 15:52:51.963979 (+ 29us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:51.964149 (+ 170us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1347,"raft.run_cpu_time_us":121,"raft.run_wall_time_us":120,"thread_start_us":50509,"threads_started":1,"wal-append.queue_time_us":106855}
I20260809 15:52:52.324214 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:52.324791 10922 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:51.965801 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.020440 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.330650 10843 raft_consensus.cc:2454] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:52.331220 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.331445 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.332556 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.332701 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.333181 10844 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:52.333374 10843 raft_consensus.cc:2454] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:52.023201 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.024364 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.030627 14139 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:52.334771 10841 raft_consensus.cc:2379] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 10: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:52.334664 14139 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:52.335594 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.336707 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:52.336858 14139 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:52.337347 10841 raft_consensus.cc:2362] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 8. Re-sending same reply.
I20260809 15:52:52.337358 10922 raft_consensus.cc:2454] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 8.
I20260809 15:52:52.033241 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.031718 14106 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:52.338780 11606 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:52.339644 14106 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:51.492599 10842 rpcz_store.cc:276] 0809 15:52:45.911013 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:45.911163 (+ 150us) service_pool.cc:227] Handling call
0809 15:52:45.911498 (+ 335us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:45.911964 (+ 466us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 15:52:45.954770 (+ 42806us) log.cc:842] Serialized 14 byte log entry
0809 15:52:46.644530 (+689760us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:46.645728 (+ 1198us) log.cc:842] Serialized 26 byte log entry
0809 15:52:46.645853 (+ 125us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:46.645881 (+ 28us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:46.645941 (+ 60us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:50.023390 (+3377449us) raft_consensus.cc:1671] finished
0809 15:52:50.023420 (+ 30us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:50.023598 (+ 178us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":28139,"prepare.queue_time_us":42589,"prepare.run_cpu_time_us":52,"prepare.run_wall_time_us":52,"raft.queue_time_us":120506,"raft.run_cpu_time_us":321,"raft.run_wall_time_us":7150,"thread_start_us":632563,"threads_started":1,"wal-append.queue_time_us":1032268,"wal-append.run_cpu_time_us":58474,"wal-append.run_wall_time_us":4168014}
I20260809 15:52:52.339862 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:52.340638 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.341385 10842 raft_consensus.cc:2379] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 7: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:52.341809 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:52.342123 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.343190 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:52.044600 13971 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:52.343815 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:52.344565 13971 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:52.050510 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:52.055164 10809 service_pool.cc:137] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45594 dropped due to backpressure. The service queue is full; it has 50 items.
I20260809 15:52:52.346073 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:52.346465 10837 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
W20260809 15:52:52.083813 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 344770us
I20260809 15:52:52.093016 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.347903 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:52.348456 10840 raft_consensus.cc:2454] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:52.095733 13992 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:52.106622 10927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 1983) took 2066 ms. Trace:
W20260809 15:52:52.108263 11014 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39978 (request call id 634) took 2684 ms (client timeout 1527 ms). Trace:
I20260809 15:52:52.111804 13917 raft_consensus.cc:3037] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:52.116374 13820 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:52.117899 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.122355 13973 consensus_queue.cc:260] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:52.139999 14047 raft_consensus.cc:3032] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:52.152110 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:52.154155 14053 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:52.169095 13959 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:52.169946 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.175416 10929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 2049) took 1146 ms. Trace:
I20260809 15:52:51.633242 10920 rpcz_store.cc:276] 0809 15:52:50.345845 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.164534 (+818689us) service_pool.cc:227] Handling call
0809 15:52:51.495829 (+331295us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.186558 10808 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:54342 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2013)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2254)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 769)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2015)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2020)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2012)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2023)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2010)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2255)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2011)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2256)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2016)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2251)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2262)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2017)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2018)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2021)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2019)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2261)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2026)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2022)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2258)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2027)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2250)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2024)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2009)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2014)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2260)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2025)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 764)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 772)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2259)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 768)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 767)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45578 (request call id 2008)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 760)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 762)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2247)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2248)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2249)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2252)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2253)
W20260809 15:52:52.196126 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 161687us
I20260809 15:52:52.202883 10891 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:51.731364 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 54239.1us
I20260809 15:52:52.205173 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:51.714310 10930 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
I20260809 15:52:51.688373 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.214895 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:52.215818 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:52.217793 14123 raft_consensus.cc:3037] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:52.218525 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:52.219295 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:52.220993 13982 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:52.225425 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:51.999158 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:52.239930 14033 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:52.267794 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.269063 11017 rpcz_store.cc:276] 0809 15:52:49.980056 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.466647 (+1486591us) service_pool.cc:227] Handling call
0809 15:52:51.687537 (+220890us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.261899 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.274322 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.275149 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:52.278044 10734 rpcz_store.cc:275] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33054 (request call id 393) took 11134 ms. Trace:
I20260809 15:52:52.293824 11011 raft_consensus.cc:2454] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:52.294639 11016 raft_consensus.cc:2454] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:52.287959 11634 rpcz_store.cc:276] 0809 15:52:44.369351 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:45.850161 (+1480810us) service_pool.cc:227] Handling call
0809 15:52:47.548347 (+1698186us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:47.744483 (+196136us) raft_consensus.cc:1520] Early marking committed up to index 7
0809 15:52:51.461489 (+3717006us) log.cc:842] Serialized 14 byte log entry
0809 15:52:51.672794 (+211305us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:51.677955 (+ 5161us) log.cc:842] Serialized 26 byte log entry
0809 15:52:51.678199 (+ 244us) raft_consensus.cc:1631] Marking committed up to 8
0809 15:52:51.678980 (+ 781us) log.cc:842] Serialized 14 byte log entry
0809 15:52:51.679432 (+ 452us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:51.679527 (+ 95us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:51.769958 (+ 90431us) raft_consensus.cc:1671] finished
0809 15:52:51.769983 (+ 25us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:51.772127 (+ 2144us) spinlock_profiling.cc:243] Waited 1.69 s on lock 0x7b5400533750. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:51.772303 (+ 176us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":150594,"prepare.queue_time_us":230,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":43,"raft.queue_time_us":151925,"raft.run_cpu_time_us":444,"raft.run_wall_time_us":445,"spinlock_wait_cycles":3884407040,"thread_start_us":192027,"threads_started":1,"wal-append.queue_time_us":273188}
I20260809 15:52:52.296689 11632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41488 (request call id 909) took 1102 ms. Trace:
I20260809 15:52:52.297336 11636 raft_consensus.cc:1226] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 7. Current term is 8. Ops: []
I20260809 15:52:52.298600 11635 raft_consensus.cc:2454] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:52.305159 11640 raft_consensus.cc:2362] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4. Re-sending same reply.
I20260809 15:52:52.306924 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:51.739459 13628 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:52.308624 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.309566 11637 raft_consensus.cc:2396] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260809 15:52:52.302798 10736 catalog_manager.cc:5495] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 reported cstate change: leader changed from <none> to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 7 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:52.311733 13656 raft_consensus.cc:729] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:52.312016 14154 tablet_replica.cc:330] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa: stopping tablet replica
I20260809 15:52:52.317544 13299 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:52.313431 13893 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:52.334060 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.337651 11018 raft_consensus.cc:2396] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260809 15:52:51.810192 14087 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:52.346580 10809 service_pool.cc:140] UpdateConsensus request on kudu.consensus.ConsensusService from 127.0.0.1:45594 dropped due to backpressure. The service queue is full; it has 50 items. Contents of service queue:
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2251)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2262)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2017)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 766)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2018)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2021)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 770)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2019)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2261)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2026)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2022)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 771)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2258)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2027)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2250)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2024)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2009)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2014)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 765)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2260)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2025)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 764)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 772)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2259)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 768)
Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 767)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45578 (request call id 2008)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 758)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 759)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 760)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 761)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 762)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 763)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2247)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2248)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2249)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2252)
Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2253)
I20260809 15:52:52.416548 14047 raft_consensus.cc:729] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:52.349776 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.416976 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.417380 10843 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:52.417693 11635 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:52.349359 10927 rpcz_store.cc:276] 0809 15:52:50.040295 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:50.040465 (+ 170us) service_pool.cc:227] Handling call
0809 15:52:50.141905 (+101440us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:51.202644 (+1060739us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:52.106245 (+903601us) spinlock_profiling.cc:243] Waited 328 ms on lock 0x7b54003ed854. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:52.106459 (+ 214us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":755287296}
I20260809 15:52:52.418605 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.419148 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:52.419445 13973 raft_consensus.cc:3037] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:52.419742 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.420534 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:52.420974 10843 raft_consensus.cc:2454] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:52.356338 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.422856 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:52.423985 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:52.424978 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:52.427919 14087 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:52.428928 10924 raft_consensus.cc:3032] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:52.428989 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.429405 10924 raft_consensus.cc:729] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:52.429698 11016 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:52.431499 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.431886 14030 consensus_queue.cc:1046] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 15:52:52.433087 14030 raft_consensus.cc:3032] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:52:52.433563 14030 raft_consensus.cc:729] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:52.434197 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.435437 13982 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:52.435371 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.435153 14030 consensus_queue.cc:260] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:52.436172 11640 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 9.
I20260809 15:52:52.436448 11015 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:52.436911 14030 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:52.437086 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.437461 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:52.438607 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.439194 11015 raft_consensus.cc:2454] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:52.440013 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 10 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.440711 11015 raft_consensus.cc:2379] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 10: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:52.350607 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.443122 11011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39978 (request call id 637) took 1883 ms. Trace:
I20260809 15:52:52.443325 10929 rpcz_store.cc:276] 0809 15:52:51.029328 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.593386 (+564058us) service_pool.cc:227] Handling call
0809 15:52:52.175344 (+581958us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.444150 11009 raft_consensus.cc:3037] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:52.383960 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.409085 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.445765 11013 raft_consensus.cc:2396] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 6 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260809 15:52:52.443526 11011 rpcz_store.cc:276] 0809 15:52:50.559379 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.676492 (+1117113us) service_pool.cc:227] Handling call
0809 15:52:52.443065 (+766573us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.433020 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.443475 11010 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:52.446914 11010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50542 (request call id 2360) took 1394 ms. Trace:
I20260809 15:52:52.447327 11010 rpcz_store.cc:276] 0809 15:52:51.052893 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.677518 (+624625us) service_pool.cc:227] Handling call
0809 15:52:52.446861 (+769343us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.419245 11635 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 8.
I20260809 15:52:52.448180 10930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54224 (request call id 1996) took 1270 ms. Trace:
I20260809 15:52:52.430851 11606 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:52.417831 14047 consensus_queue.cc:260] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:52.438001 11640 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:52.406315 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:52.441546 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.424134 10736 catalog_manager.cc:5495] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 3 to 4, leader changed from c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 4 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260809 15:52:52.349649 11014 rpcz_store.cc:269] 0809 15:52:49.423757 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:50.559813 (+1136056us) service_pool.cc:227] Handling call
0809 15:52:52.108071 (+1548258us) spinlock_profiling.cc:243] Waited 83.9 ms on lock 0x7b540019ebd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:52.108197 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":193032960}
I20260809 15:52:52.355962 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:52.422029 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.453685 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.454048 11604 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:52.454427 11012 raft_consensus.cc:2454] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:52.454573 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:52.454617 13925 raft_consensus.cc:2781] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader election won for term 8
W20260809 15:52:52.462400 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 111993us
I20260809 15:52:52.455657 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.495205 11634 raft_consensus.cc:2379] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 6: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:52.495783 10837 raft_consensus.cc:2454] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:52.496402 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.496832 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.497414 10837 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:52.497848 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.498658 10844 raft_consensus.cc:2379] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 5: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:52.499228 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.499485 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.448540 10930 rpcz_store.cc:276] 0809 15:52:51.182453 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.619339 (+436886us) service_pool.cc:227] Handling call
0809 15:52:52.448122 (+828783us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.461325 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.421136 10927 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:52.491282 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.458864 10922 raft_consensus.cc:2454] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 2.
I20260809 15:52:52.490531 10931 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:52.492692 10838 raft_consensus.cc:2454] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:52.492895 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.493129 10841 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:52.493572 10842 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:52.494062 11015 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:52.493835 13656 consensus_queue.cc:260] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:52.503187 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 300402us
I20260809 15:52:52.494397 13893 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:52.463462 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:52.500453 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.351464 13971 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:52.521152 10847 raft_consensus.cc:2454] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:52.525859 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.462158 10921 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:54216
I20260809 15:52:52.541338 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.523767 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.524222 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.524417 10843 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 7.
I20260809 15:52:52.542055 10848 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 10.
I20260809 15:52:52.530956 11641 raft_consensus.cc:2454] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:52.501606 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:52.502648 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 71903.6us
I20260809 15:52:52.525080 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.528793 11017 raft_consensus.cc:2454] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:52.529875 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.532603 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.520948 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.521013 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.543843 10838 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:52.545491 11636 raft_consensus.cc:2454] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 2.
I20260809 15:52:52.544328 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.525151 14047 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:52.462842 13299 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:52.540741 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 86795.6us
I20260809 15:52:52.547945 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.548779 11604 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:52.548959 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.540998 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.501258 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.543463 11606 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:52.549674 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:52.545908 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.537840 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.546545 10891 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:52.546471 11013 raft_consensus.cc:2379] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 2: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:52.546389 11016 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:52.541776 10924 consensus_queue.cc:260] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:52.545513 14031 raft_consensus.cc:2781] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:52.553133 10838 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:52.553318 14031 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:52.553716 14031 raft_consensus.cc:3037] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:52.549551 10921 raft_consensus.cc:595] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Received request to transfer leadership
I20260809 15:52:52.534947 13925 raft_consensus.cc:686] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [term 8 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:52.545884 13958 raft_consensus.cc:2781] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:52.457147 11632 rpcz_store.cc:276] 0809 15:52:51.199529 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:51.226866 (+ 27337us) service_pool.cc:227] Handling call
0809 15:52:52.296448 (+1069582us) spinlock_profiling.cc:243] Waited 273 ms on lock 0x7b54005203d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:52.296610 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":628226304}
I20260809 15:52:52.551105 11639 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 9.
I20260809 15:52:52.549742 13656 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:52.550398 11011 raft_consensus.cc:2454] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:52.547739 14033 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:52.554057 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 62493.4us
I20260809 15:52:52.554757 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:52:52.575145 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 133872us
I20260809 15:52:52.556172 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.556401 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.560258 10924 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:52.562288 11635 raft_consensus.cc:2454] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 3.
I20260809 15:52:52.564401 11015 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:52.567589 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.561779 10844 raft_consensus.cc:2454] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:52.569501 11011 raft_consensus.cc:1226] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:52.580571 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.573467 11013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2333) took 3052 ms. Trace:
I20260809 15:52:52.573992 11014 raft_consensus.cc:2396] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 7 because replica has last-logged OpId of term: 4 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 2.
I20260809 15:52:52.554510 11012 raft_consensus.cc:2396] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 6 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 15:52:52.557036 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.558138 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.582830 11641 raft_consensus.cc:2454] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 1.
I20260809 15:52:52.567243 11639 raft_consensus.cc:1261] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260809 15:52:52.568096 10891 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:52.581332 11013 rpcz_store.cc:276] 0809 15:52:49.521062 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.572655 (+3051593us) service_pool.cc:227] Handling call
0809 15:52:52.572963 (+ 308us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:52.573165 (+ 202us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:52.573186 (+ 21us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:52.573228 (+ 42us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:52.573264 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:52.573310 (+ 46us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:52.573396 (+ 86us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.569543 10809 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:52.573614 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.587929 11009 raft_consensus.cc:2396] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 5 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260809 15:52:52.554242 14005 raft_consensus.cc:2781] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260809 15:52:52.592700 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 44034.9us
I20260809 15:52:52.589641 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:52.571275 11640 raft_consensus.cc:2454] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
I20260809 15:52:52.591173 11011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2335) took 3068 ms. Trace:
I20260809 15:52:52.585464 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.581149 10837 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:52:52.594105 11011 rpcz_store.cc:276] 0809 15:52:49.522772 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.568958 (+3046186us) service_pool.cc:227] Handling call
0809 15:52:52.569318 (+ 360us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:52.590914 (+ 21596us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:52.591120 (+ 206us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.446874 10734 rpcz_store.cc:276] 0809 15:52:41.144029 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:41.145278 (+ 1249us) service_pool.cc:227] Handling call
0809 15:52:52.277817 (+11132539us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 15:52:44.713105 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 15:52:44.799334 (+ 86229us) write_op.cc:432] Acquiring schema lock in shared mode
0809 15:52:44.799383 (+ 49us) write_op.cc:435] Acquired schema lock
0809 15:52:44.799403 (+ 20us) tablet.cc:660] Decoding operations
0809 15:52:44.800701 (+ 1298us) write_op.cc:620] Acquiring the partition lock for write op
0809 15:52:44.800780 (+ 79us) write_op.cc:641] Partition lock acquired for write op
0809 15:52:44.800816 (+ 36us) tablet.cc:683] Acquiring locks for 11 operations
0809 15:52:44.811803 (+ 10987us) tablet.cc:699] Row locks acquired
0809 15:52:44.811832 (+ 29us) write_op.cc:260] PREPARE: finished
0809 15:52:44.830552 (+ 18720us) write_op.cc:270] Start()
0809 15:52:44.836451 (+ 5899us) write_op.cc:276] Timestamp: P: 1786290764823402 usec, L: 0
0809 15:52:44.836481 (+ 30us) op_driver.cc:331] REPLICATION: starting
0809 15:52:44.930653 (+ 94172us) log.cc:842] Serialized 4554 byte log entry
0809 15:52:45.338418 (+407765us) op_driver.cc:444] REPLICATION: finished
0809 15:52:46.225887 (+887469us) write_op.cc:301] APPLY: starting
0809 15:52:46.234727 (+ 8840us) tablet.cc:1365] starting BulkCheckPresence
0809 15:52:46.326157 (+ 91430us) tablet.cc:1368] finished BulkCheckPresence
0809 15:52:46.326297 (+ 140us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 15:52:46.341000 (+ 14703us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 15:52:46.341229 (+ 229us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=11
0809 15:52:46.341274 (+ 45us) write_op.cc:312] APPLY: finished
0809 15:52:46.341543 (+ 269us) log.cc:842] Serialized 83 byte log entry
0809 15:52:47.335360 (+993817us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:52:47.356927 (+ 21567us) write_op.cc:454] Released schema lock
0809 15:52:47.473017 (+116090us) write_op.cc:341] FINISH: Updating metrics
0809 15:52:47.659041 (+186024us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:52:47.665961 (+ 6920us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":4322,"reported_tablets":27,"child_traces":[["op",{"apply.queue_time_us":780629,"apply.run_cpu_time_us":45036,"apply.run_wall_time_us":1664012,"num_ops":11,"prepare.queue_time_us":372890,"prepare.run_cpu_time_us":12153,"prepare.run_wall_time_us":435651,"raft.queue_time_us":118712,"raft.run_cpu_time_us":34962,"raft.run_wall_time_us":503425,"replication_time_us":496398,"thread_start_us":1751853,"threads_started":5,"wal-append.queue_time_us":1373433,"wal-append.run_cpu_time_us":62911,"wal-append.run_wall_time_us":4527008}]]}
I20260809 15:52:52.588994 10842 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:52.590350 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.589730 11637 raft_consensus.cc:2379] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 4: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:52.591355 10811 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:52.599331 10848 raft_consensus.cc:2396] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260809 15:52:52.599720 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.600878 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:52.601482 10843 raft_consensus.cc:2454] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:52.573102 10736 catalog_manager.cc:5495] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 2 to 3, leader changed from c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 3 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: HEALTHY } } }
I20260809 15:52:52.604059 11017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 633) took 3189 ms. Trace:
I20260809 15:52:52.568293 13982 raft_consensus.cc:2781] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:52.559448 11632 raft_consensus.cc:1226] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:52.604514 11017 rpcz_store.cc:276] 0809 15:52:49.407583 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.547554 (+3139971us) service_pool.cc:227] Handling call
0809 15:52:52.573530 (+ 25976us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:52.590051 (+ 16521us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:52.590121 (+ 70us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:52.596383 (+ 6262us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:52.596431 (+ 48us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:52.596481 (+ 50us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:52.596580 (+ 99us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.607012 11604 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:52.572578 14095 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:52.561617 13299 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:52.565604 14087 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:52:52.609836 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 66457.6us
I20260809 15:52:52.559080 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.613879 10809 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:52.611467 13806 raft_consensus.cc:2781] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:52.595386 13971 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Advancing to term 10
I20260809 15:52:52.595249 14004 raft_consensus.cc:2726] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:52.592343 14005 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:52.618171 11014 raft_consensus.cc:1261] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 15:52:52.625742 11014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2339) took 2746 ms. Trace:
W20260809 15:52:52.621177 10893 connection.cc:232] RPC call timeout handler was delayed by -1.8678310510003939s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1791s for a response
I20260809 15:52:52.626102 11014 rpcz_store.cc:276] 0809 15:52:49.879639 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.590426 (+2710787us) service_pool.cc:227] Handling call
0809 15:52:52.615417 (+ 24991us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:52.625543 (+ 10126us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:52.625696 (+ 153us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:52:52.626437 10893 connection.cc:232] RPC call timeout handler was delayed by -1.7994310510111973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
W20260809 15:52:52.626816 10893 connection.cc:232] RPC call timeout handler was delayed by -1.7994310510111973s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1867s for a response
W20260809 15:52:52.627133 10893 connection.cc:232] RPC call timeout handler was delayed by -1.5987310510245152s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.209s for a response
W20260809 15:52:52.627409 10893 connection.cc:232] RPC call timeout handler was delayed by -1.5069310510298237s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21920000000000003s for a response
W20260809 15:52:52.627698 10893 connection.cc:232] RPC call timeout handler was delayed by -1.0848310510045849s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2661s for a response
I20260809 15:52:52.618515 10837 raft_consensus.cc:1226] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260809 15:52:52.609588 13982 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:52.619663 11016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2334) took 3097 ms. Trace:
I20260809 15:52:52.618161 10736 catalog_manager.cc:5495] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 3 to 7, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 7 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:52.631815 11015 raft_consensus.cc:1261] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 15:52:52.619179 14087 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:52.631675 11016 rpcz_store.cc:276] 0809 15:52:49.521877 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.552378 (+3030501us) service_pool.cc:227] Handling call
0809 15:52:52.609184 (+ 56806us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:52.619259 (+ 10075us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:52.619310 (+ 51us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:52.619357 (+ 47us) raft_consensus.cc:1631] Marking committed up to 3
0809 15:52:52.619393 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:52.619464 (+ 71us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:52.619572 (+ 108us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.632433 11015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2336) took 3108 ms. Trace:
I20260809 15:52:52.634953 11642 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:52.635296 11011 raft_consensus.cc:1226] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:52.635933 11642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41478 (request call id 987) took 1017 ms. Trace:
I20260809 15:52:52.635624 11015 rpcz_store.cc:276] 0809 15:52:49.524351 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.578700 (+3054349us) service_pool.cc:227] Handling call
0809 15:52:52.611782 (+ 33082us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:52.632227 (+ 20445us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:52.632389 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.570261 14033 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } }
I20260809 15:52:52.637373 10841 raft_consensus.cc:1261] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 5 index: 3. (index mismatch)
I20260809 15:52:52.631924 11604 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:52.636340 11642 rpcz_store.cc:276] 0809 15:52:51.629396 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.151739 (+522343us) service_pool.cc:227] Handling call
0809 15:52:52.635879 (+484140us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.640326 10809 leader_election.cc:302] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:52.570377 14031 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:52.641453 11012 raft_consensus.cc:1226] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 1. Current term is 2. Ops: []
I20260809 15:52:52.641881 14049 raft_consensus.cc:2781] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:52.641986 14154 raft_consensus.cc:2229] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Raft consensus shutting down.
W20260809 15:52:52.652710 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 75246us
I20260809 15:52:52.642433 13979 raft_consensus.cc:2781] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:52.639143 13806 raft_consensus.cc:686] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
W20260809 15:52:52.645280 14156 log.cc:924] Time spent T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.105s user 0.000s sys 0.001s
I20260809 15:52:52.639303 13628 raft_consensus.cc:2781] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:52.646848 13853 raft_consensus.cc:2726] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 15:52:52.648100 11011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2342) took 2669 ms. Trace:
I20260809 15:52:52.657922 11011 rpcz_store.cc:276] 0809 15:52:49.979008 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.606182 (+2627174us) service_pool.cc:227] Handling call
0809 15:52:52.635108 (+ 28926us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:52.647922 (+ 12814us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:52.648059 (+ 137us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.658227 11606 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:52.619791 13958 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:52.649652 11012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2340) took 2680 ms. Trace:
I20260809 15:52:52.649199 10891 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:52.653373 10809 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:52.653788 10837 raft_consensus.cc:1226] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:52.557786 13925 consensus_queue.cc:237] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:52.631090 14005 raft_consensus.cc:3037] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:52.660634 11012 rpcz_store.cc:276] 0809 15:52:49.969485 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.602774 (+2633289us) service_pool.cc:227] Handling call
0809 15:52:52.623269 (+ 20495us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:52.649404 (+ 26135us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:52.649592 (+ 188us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.656858 10811 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:52.654055 11013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2337) took 3129 ms. Trace:
I20260809 15:52:52.661548 11017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2343) took 2682 ms. Trace:
I20260809 15:52:52.661620 11013 rpcz_store.cc:276] 0809 15:52:49.525051 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.593938 (+3068887us) service_pool.cc:227] Handling call
0809 15:52:52.617687 (+ 23749us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:52.653822 (+ 36135us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:52.653848 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:52.653888 (+ 40us) raft_consensus.cc:1631] Marking committed up to 3
0809 15:52:52.653907 (+ 19us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:52.653942 (+ 35us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:52.654015 (+ 73us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.636363 13982 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:52.663282 11604 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:52.661985 11017 rpcz_store.cc:276] 0809 15:52:49.979385 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.607046 (+2627661us) service_pool.cc:227] Handling call
0809 15:52:52.629706 (+ 22660us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:52.661231 (+ 31525us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:52.661257 (+ 26us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:52.661313 (+ 56us) raft_consensus.cc:1631] Marking committed up to 2
0809 15:52:52.661345 (+ 32us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:52.661403 (+ 58us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:52.661497 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.662581 11014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2344) took 2682 ms. Trace:
I20260809 15:52:52.664836 14154 raft_consensus.cc:2258] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Raft consensus is shut down!
I20260809 15:52:52.667032 11014 rpcz_store.cc:276] 0809 15:52:49.979717 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.626922 (+2647205us) service_pool.cc:227] Handling call
0809 15:52:52.657317 (+ 30395us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:52.662281 (+ 4964us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:52.662305 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:52.662344 (+ 39us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:52.662388 (+ 44us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:52.662441 (+ 53us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:52.662532 (+ 91us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.669775 10809 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:52.658070 11634 raft_consensus.cc:2396] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260809 15:52:52.652710 14049 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:52.621491 14095 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:52.666719 10736 catalog_manager.cc:5495] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 2 to 3, leader changed from c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 3 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:52.677413 11012 raft_consensus.cc:1226] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:52.656597 14133 raft_consensus.cc:2781] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:52.654466 13979 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:52.660838 13906 raft_consensus.cc:2781] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:52.665711 13958 raft_consensus.cc:3037] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:52.688341 10891 leader_election.cc:302] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:52.690210 13825 raft_consensus.cc:2781] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:52.690277 10840 raft_consensus.cc:2454] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:52.682627 13999 raft_consensus.cc:2781] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:52.679127 13893 raft_consensus.cc:2781] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:52.682395 11604 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:52.683007 11606 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:52.693426 10736 catalog_manager.cc:5495] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 4 to 5, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 5 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:52.665464 13628 raft_consensus.cc:686] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:52.701327 11010 raft_consensus.cc:2396] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 5 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260809 15:52:52.676133 14049 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:52.706347 11016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 635) took 2147 ms. Trace:
W20260809 15:52:52.735149 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 61912us
W20260809 15:52:52.739530 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 51310.9us
I20260809 15:52:52.694751 13906 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:52.683607 13979 raft_consensus.cc:3037] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:52.750960 11012 raft_consensus.cc:1226] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260809 15:52:52.704918 13893 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:52.760986 11606 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:52.680467 14133 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:52.771153 10736 catalog_manager.cc:5495] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 5 to 6, leader changed from c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 6 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:52.700835 14030 raft_consensus.cc:2781] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:52.767431 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.789934 11640 raft_consensus.cc:2454] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:52.750185 11016 rpcz_store.cc:276] 0809 15:52:50.558592 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.650358 (+2091766us) service_pool.cc:227] Handling call
0809 15:52:52.671687 (+ 21329us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:52.700618 (+ 28931us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:52.700661 (+ 43us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:52.706002 (+ 5341us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:52.706059 (+ 57us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:52.706149 (+ 90us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:52.706281 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:52.700364 13825 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:52.795951 11018 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:52.796566 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.798898 10891 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:52.799227 14154 ts_tablet_manager.cc:1905] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa: Deleting tablet data with delete state TABLET_DATA_TOMBSTONED
I20260809 15:52:52.800181 14087 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:52.766628 13806 consensus_queue.cc:237] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:52.767122 13628 consensus_queue.cc:237] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:52.750185 14123 raft_consensus.cc:2726] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:52.783216 13893 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:52.831390 11018 raft_consensus.cc:1261] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260809 15:52:52.805328 14005 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:52.834256 10770 debug-util.cc:398] Leaking SignalData structure 0x7b08001f0580 after lost signal to thread 9556
I20260809 15:52:52.835618 14005 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:52.836318 14005 raft_consensus.cc:2726] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:52.837157 14005 raft_consensus.cc:2726] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:52.763204 13906 raft_consensus.cc:3037] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:52.701476 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:52.851830 14033 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:52.700698 13999 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:52.852424 13999 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:52.812677 10924 raft_consensus.cc:1261] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260809 15:52:52.813647 14048 raft_consensus.cc:2781] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:52.770046 13798 raft_consensus.cc:2781] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:52.782882 13913 raft_consensus.cc:2781] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:52.786670 14133 raft_consensus.cc:3037] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:52.828579 11009 raft_consensus.cc:1261] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260809 15:52:52.837371 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:52.835421 10770 debug-util.cc:398] Leaking SignalData structure 0x7b0800165ca0 after lost signal to thread 10773
W20260809 15:52:52.858203 10770 debug-util.cc:398] Leaking SignalData structure 0x7b080009f020 after lost signal to thread 10875
W20260809 15:52:52.858593 10770 debug-util.cc:398] Leaking SignalData structure 0x7b0800042260 after lost signal to thread 10877
W20260809 15:52:52.859880 10770 debug-util.cc:398] Leaking SignalData structure 0x7b080037c800 after lost signal to thread 10959
W20260809 15:52:52.860199 10770 debug-util.cc:398] Leaking SignalData structure 0x7b08003cf040 after lost signal to thread 10960
I20260809 15:52:52.843269 11010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39978 (request call id 640) took 1152 ms. Trace:
I20260809 15:52:52.861460 11010 rpcz_store.cc:276] 0809 15:52:51.690648 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.465968 (+775320us) service_pool.cc:227] Handling call
0809 15:52:52.828086 (+362118us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b540004bfd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:52.843197 (+ 15111us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":358759680}
W20260809 15:52:52.863721 10770 debug-util.cc:398] Leaking SignalData structure 0x7b08003f5540 after lost signal to thread 11047
W20260809 15:52:52.864203 10770 debug-util.cc:398] Leaking SignalData structure 0x7b08003f5520 after lost signal to thread 11048
W20260809 15:52:52.865654 10770 debug-util.cc:398] Leaking SignalData structure 0x7b08004246a0 after lost signal to thread 11667
I20260809 15:52:52.854549 10924 pending_rounds.cc:76] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3: Aborting all ops after (but not including) 5
I20260809 15:52:52.812070 13825 raft_consensus.cc:3037] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:52.789659 14030 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:52.856652 11009 pending_rounds.cc:76] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa: Aborting all ops after (but not including) 4
I20260809 15:52:52.867022 11637 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:52.869163 10924 pending_rounds.cc:97] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260809 15:52:52.868582 11016 raft_consensus.cc:1226] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:52.876847 11009 pending_rounds.cc:97] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260809 15:52:52.881111 10924 raft_consensus.cc:2866] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:52.856137 14048 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:52.854663 10920 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:54216
I20260809 15:52:52.883355 11009 raft_consensus.cc:2866] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:52.883912 13992 raft_consensus.cc:2781] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:52.888497 10736 catalog_manager.cc:5495] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 2 to 4, leader changed from c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 4 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:52.857378 13913 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:52.857547 13798 raft_consensus.cc:482] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:52.878599 14030 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:52.893383 10920 raft_consensus.cc:595] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Received request to transfer leadership
I20260809 15:52:52.882444 13971 raft_consensus.cc:2726] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 15:52:52.880749 14031 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:52.888002 14048 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:52.909508 11639 raft_consensus.cc:1261] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260809 15:52:52.890921 13992 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:52.894164 13798 raft_consensus.cc:3037] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:52.893952 13913 raft_consensus.cc:3037] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:52.904801 10736 catalog_manager.cc:5495] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 1 to 3, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 3 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:52.899683 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
W20260809 15:52:52.942320 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 182497us
I20260809 15:52:52.946866 11606 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:52.928720 13992 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:52.986295 10736 catalog_manager.cc:5495] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 2 to 3, leader changed from c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 3 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:52.997433 11606 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
W20260809 15:52:52.998659 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 65403.2us
I20260809 15:52:52.984620 11632 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:53.010200 11606 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:53.015771 11015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 638) took 1953 ms. Trace:
I20260809 15:52:53.017843 11009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2338) took 3470 ms. Trace:
W20260809 15:52:53.018002 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 219185us
I20260809 15:52:53.018350 11009 rpcz_store.cc:276] 0809 15:52:49.533020 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.588523 (+3055503us) service_pool.cc:227] Handling call
0809 15:52:52.828228 (+239705us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:52.995250 (+167022us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:53.002754 (+ 7504us) spinlock_profiling.cc:243] Waited 227 ms on lock 0x7b540004bfd0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:53.002896 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":521694592}
I20260809 15:52:52.997623 13971 raft_consensus.cc:2781] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:52:53.016330 11015 rpcz_store.cc:276] 0809 15:52:51.049439 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.647762 (+1598323us) service_pool.cc:227] Handling call
0809 15:52:52.657028 (+ 9266us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:52.674127 (+ 17099us) raft_consensus.cc:1520] Early marking committed up to index 6
0809 15:52:52.856357 (+182230us) log.cc:842] Serialized 14 byte log entry
0809 15:52:53.001662 (+145305us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:53.001730 (+ 68us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:53.001795 (+ 65us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:53.001897 (+ 102us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:53.002041 (+ 144us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1284,"prepare.queue_time_us":266615,"raft.queue_time_us":16142,"thread_start_us":280761,"threads_started":2}
I20260809 15:52:53.022241 10736 catalog_manager.cc:5495] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 reported cstate change: term changed from 2 to 3, leader changed from c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 3 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:52:53.031571 11606 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
W20260809 15:52:53.024456 13973 consensus_meta.cc:287] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.604s user 0.011s sys 0.001s
W20260809 15:52:53.039729 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 357406us
I20260809 15:52:53.043536 10839 raft_consensus.cc:2396] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260809 15:52:53.025177 14005 raft_consensus.cc:2781] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:53.049032 11606 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:53.030158 14004 raft_consensus.cc:2781] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
W20260809 15:52:53.053182 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 383455us
I20260809 15:52:53.028108 13971 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:53.066458 10839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 753) took 1040 ms. Trace:
I20260809 15:52:53.042019 14095 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:53.052277 14005 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:53.047681 14094 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:53.055577 14004 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:53.058637 13971 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:53.037963 14049 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.075250 11606 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:53.067786 10839 rpcz_store.cc:276] 0809 15:52:52.021961 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.022672 (+ 711us) service_pool.cc:227] Handling call
0809 15:52:53.061088 (+1038416us) spinlock_profiling.cc:243] Waited 709 ms on lock 0x7b540037e6d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:53.062420 (+ 1332us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1630064896}
I20260809 15:52:53.083588 13893 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.077720 14094 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.087242 13982 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:53.073189 14005 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:53.102270 14048 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:53.109184 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 46569.9us
I20260809 15:52:53.078541 14004 raft_consensus.cc:3037] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:53.126169 11013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2096) took 1101 ms. Trace:
I20260809 15:52:53.109360 13992 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.134150 13973 consensus_queue.cc:1046] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:53.135229 13973 raft_consensus.cc:3032] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:53.135620 13973 raft_consensus.cc:729] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:53.119693 13299 raft_consensus.cc:2726] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:53.135735 14033 consensus_queue.cc:1046] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260809 15:52:53.136765 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 479941us
I20260809 15:52:53.136586 13973 consensus_queue.cc:260] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:53.138347 11017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2097) took 1113 ms. Trace:
I20260809 15:52:53.138563 13992 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:53.138759 10843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2249) took 1092 ms. Trace:
I20260809 15:52:53.138862 13893 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:53.139196 10843 rpcz_store.cc:276] 0809 15:52:52.046183 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.629958 (+583775us) service_pool.cc:227] Handling call
0809 15:52:53.061250 (+431292us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:53.126122 (+ 64872us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 15:52:53.126151 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:53.126201 (+ 50us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:53.126240 (+ 39us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:53.138044 (+ 11804us) spinlock_profiling.cc:243] Waited 58.6 ms on lock 0x7b540037e6d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:53.138115 (+ 71us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:53.138599 (+ 484us) spinlock_profiling.cc:243] Waited 431 ms on lock 0x7b540037e6d0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:53.138708 (+ 109us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5853,"raft.queue_time_us":8309,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":75,"spinlock_wait_cycles":1125932160}
W20260809 15:52:53.144028 10811 connection.cc:232] RPC call timeout handler was delayed by -1.900547783006914s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15080000000000002s for a response
W20260809 15:52:53.144971 10811 connection.cc:232] RPC call timeout handler was delayed by -1.4793477830244228s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1976s for a response
W20260809 15:52:53.145282 10811 connection.cc:232] RPC call timeout handler was delayed by -1.3668477830360644s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2101s for a response
W20260809 15:52:53.145663 10811 connection.cc:232] RPC call timeout handler was delayed by -1.3560477830469608s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.21130000000000002s for a response
W20260809 15:52:53.145902 10811 connection.cc:232] RPC call timeout handler was delayed by -1.2588477830286138s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22210000000000002s for a response
I20260809 15:52:53.146687 11632 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:53.148723 14123 consensus_queue.cc:1035] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:53.150831 10838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 762) took 1114 ms. Trace:
I20260809 15:52:53.151232 10838 rpcz_store.cc:276] 0809 15:52:52.038254 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.608887 (+570633us) service_pool.cc:227] Handling call
0809 15:52:52.615784 (+ 6897us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:52.632364 (+ 16580us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:52.632391 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:52.661126 (+ 28735us) log.cc:842] Serialized 26 byte log entry
0809 15:52:53.032830 (+371704us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:53.032874 (+ 44us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:53.032958 (+ 84us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:53.135223 (+102265us) spinlock_profiling.cc:243] Waited 2.38 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:52:53.150525 (+ 15302us) raft_consensus.cc:1671] finished
0809 15:52:53.150543 (+ 18us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:53.150752 (+ 209us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4539,"raft.queue_time_us":18769,"raft.run_cpu_time_us":136,"raft.run_wall_time_us":3083,"spinlock_wait_cycles":5471104,"thread_start_us":322533,"threads_started":1,"wal-append.queue_time_us":432958}
I20260809 15:52:53.102430 10978 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:53.152772 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 75322.9us
I20260809 15:52:53.091815 14030 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.153532 10978 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:53.153363 13958 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.152871 14095 raft_consensus.cc:2781] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:53.062198 14171 raft_consensus.cc:2781] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:53.137203 14033 raft_consensus.cc:3032] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:53.099249 13906 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.157997 10809 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:53.276932 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 119050us
I20260809 15:52:53.154675 14030 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:53.168031 14172 consensus_queue.cc:1046] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 4.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:53.156941 13917 raft_consensus.cc:2781] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:53.167613 14169 consensus_queue.cc:1046] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260809 15:52:53.167318 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 58780.2us
I20260809 15:52:53.126523 13913 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.128505 11013 rpcz_store.cc:276] 0809 15:52:52.024573 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.662791 (+638218us) service_pool.cc:227] Handling call
0809 15:52:52.673969 (+ 11178us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:52.685073 (+ 11104us) raft_consensus.cc:1520] Early marking committed up to index 4
0809 15:52:52.866857 (+181784us) log.cc:842] Serialized 14 byte log entry
0809 15:52:53.125599 (+258742us) log.cc:842] Serialized 14 byte log entry
0809 15:52:53.125780 (+ 181us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:53.125824 (+ 44us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:52:53.125897 (+ 73us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:53.125967 (+ 70us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:53.126110 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8944,"prepare.queue_time_us":310441,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":44,"raft.queue_time_us":9227,"thread_start_us":397753,"threads_started":2}
I20260809 15:52:53.138713 11017 rpcz_store.cc:276] 0809 15:52:52.024947 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.666685 (+641738us) service_pool.cc:227] Handling call
0809 15:52:52.667003 (+ 318us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:52.683553 (+ 16550us) raft_consensus.cc:1520] Early marking committed up to index 4
0809 15:52:52.683610 (+ 57us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:52.703816 (+ 20206us) log.cc:842] Serialized 26 byte log entry
0809 15:52:52.923518 (+219702us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:52:52.923573 (+ 55us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:52.923660 (+ 87us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:53.138142 (+214482us) raft_consensus.cc:1671] finished
0809 15:52:53.138164 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:53.138290 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1889,"raft.queue_time_us":12097,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":128,"thread_start_us":153050,"threads_started":1,"wal-append.queue_time_us":322375}
I20260809 15:52:53.155390 10978 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:53.159133 14174 raft_consensus.cc:2726] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:52:53.160794 14048 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:53.163183 14049 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:53.178828 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 68723.4us
I20260809 15:52:53.135849 14094 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:53.179438 13893 consensus_queue.cc:1035] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 15:52:53.138018 13973 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:53.163972 14133 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:53.295331 14133 leader_election.cc:288] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:53.216778 13825 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.198984 13999 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.297286 13825 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:53.166529 13982 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:53.174587 13979 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.297293 13999 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:53.299451 13979 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:53.256397 10722 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.LeaderStepDown -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 257957us
I20260809 15:52:53.205087 13971 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.301448 13971 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:53.187364 14004 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.238797 14005 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.302968 14172 consensus_queue.cc:1035] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:53.303354 14005 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:53.268036 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 102961us
I20260809 15:52:53.268355 11014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2098) took 1240 ms. Trace:
I20260809 15:52:53.266803 14095 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:53.304244 14095 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:53.304454 14172 raft_consensus.cc:3032] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:53.305106 14172 raft_consensus.cc:729] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:53.304080 11014 rpcz_store.cc:276] 0809 15:52:52.025302 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.670182 (+644880us) service_pool.cc:227] Handling call
0809 15:52:52.680914 (+ 10732us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:52.699158 (+ 18244us) raft_consensus.cc:1520] Early marking committed up to index 8
0809 15:52:53.041670 (+342512us) log.cc:842] Serialized 14 byte log entry
0809 15:52:53.264787 (+223117us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:53.264867 (+ 80us) raft_consensus.cc:1631] Marking committed up to 8
0809 15:52:53.264934 (+ 67us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:53.265024 (+ 90us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:53.265170 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8000,"prepare.queue_time_us":313710,"prepare.run_cpu_time_us":46,"prepare.run_wall_time_us":46,"raft.queue_time_us":12343,"raft.run_cpu_time_us":99,"raft.run_wall_time_us":993,"thread_start_us":478982,"threads_started":2,"wal-append.queue_time_us":238288}
I20260809 15:52:53.270648 14171 raft_consensus.cc:686] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:53.261219 14154 ts_tablet_manager.cc:1918] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa: tablet deleted with delete type TABLET_DATA_TOMBSTONED: last-logged OpId 4.6
I20260809 15:52:53.239146 13798 raft_consensus.cc:504] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.269897 14033 raft_consensus.cc:729] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
W20260809 15:52:53.291728 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 136429us
I20260809 15:52:53.304306 11606 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:53.278789 13806 raft_consensus.cc:2781] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:53.314932 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:53.279115 13917 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:53.316342 13917 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:53.316438 14154 log.cc:1196] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa: Deleting WAL directory at /tmp/dist-test-taskhl0EAE/test-tmp/auto_leader_rebalancer-test.0.LeaderRebalancerTest.AddTserver.1786290617080470-9552-0/minicluster-data/ts-2-root/wals/5e8d12b5ef9e415cbcb25e62ab228546
I20260809 15:52:53.317921 14154 rpcz_store.cc:275] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:50526 (request call id 1174) took 1039 ms. Trace:
I20260809 15:52:53.318293 14154 rpcz_store.cc:276] 0809 15:52:52.278961 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.279206 (+ 245us) service_pool.cc:227] Handling call
0809 15:52:52.311328 (+ 32122us) ts_tablet_manager.cc:1145] Deleting tablet 5e8d12b5ef9e415cbcb25e62ab228546
0809 15:52:52.311424 (+ 96us) ts_tablet_manager.cc:1065] Acquired tablet manager lock
0809 15:52:53.088661 (+777237us) tablet_metadata.cc:641] Metadata flushed
0809 15:52:53.239238 (+150577us) tablet_metadata.cc:641] Metadata flushed
0809 15:52:53.317838 (+ 78600us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":129962,"tablet-delete.queue_time_us":31037,"thread_start_us":14089,"threads_started":1}
I20260809 15:52:53.320966 13798 leader_election.cc:288] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:53.321166 10838 raft_consensus.cc:1261] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 3 index: 4. (index mismatch)
I20260809 15:52:53.321923 14033 consensus_queue.cc:260] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.322633 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:53.323068 14033 raft_consensus.cc:3037] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:53.323377 10838 raft_consensus.cc:2379] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 7: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:53.323500 13917 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:53.324750 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:53.303882 14004 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:53.325428 13917 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:53.326534 11011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 639) took 1636 ms. Trace:
I20260809 15:52:53.326894 11011 rpcz_store.cc:276] 0809 15:52:51.690149 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.666116 (+975967us) service_pool.cc:227] Handling call
0809 15:52:52.677411 (+ 11295us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:52.738047 (+ 60636us) raft_consensus.cc:1520] Early marking committed up to index 6
0809 15:52:53.050227 (+312180us) log.cc:842] Serialized 14 byte log entry
0809 15:52:53.326182 (+275955us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:53.326241 (+ 59us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:53.326291 (+ 50us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:53.326367 (+ 76us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:53.326473 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8218,"prepare.queue_time_us":263411,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":41,"raft.queue_time_us":25733,"raft.run_cpu_time_us":106,"raft.run_wall_time_us":106,"thread_start_us":502838,"threads_started":3,"wal-append.queue_time_us":211240}
I20260809 15:52:53.331769 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:53.332978 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:53.332842 14095 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.333652 11011 raft_consensus.cc:2454] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:53.335094 14095 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:53.287814 14030 raft_consensus.cc:3032] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:53.340232 10719 catalog_manager.cc:4841] TS d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223): tablet 5e8d12b5ef9e415cbcb25e62ab228546 (table test-workload [id=ec9ea98dd99b4c8eb387ca1fcf13fcd8]) successfully deleted
I20260809 15:52:53.340535 14030 raft_consensus.cc:729] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:53.341576 14030 consensus_queue.cc:260] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.342391 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:53.342442 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:53.343919 11018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 654) took 1028 ms. Trace:
I20260809 15:52:53.345239 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:53.346480 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.347395 11012 raft_consensus.cc:1226] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260809 15:52:53.347601 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:53.347947 14004 raft_consensus.cc:2726] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:53.348239 13979 consensus_queue.cc:1035] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:52:53.349030 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:53.349622 10841 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:52:53.349822 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:53.350443 10839 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:53.350731 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:53.351279 11010 raft_consensus.cc:3037] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
W20260809 15:52:53.351588 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 47394.3us
I20260809 15:52:53.329448 14047 raft_consensus.cc:2781] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:53.355471 14047 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:53.355942 14047 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:53.355552 14171 consensus_queue.cc:237] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.358043 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:53.355058 13906 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:53.359241 11016 raft_consensus.cc:2396] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 8 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260809 15:52:53.340130 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:53.362617 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:53.363291 11606 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:53.342952 14030 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:53.365381 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:53.365767 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:53.366433 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:53.366777 10837 raft_consensus.cc:2454] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:53.365921 11015 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:53.329593 13806 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:53.373229 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:53.373448 13806 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:53.373977 10843 raft_consensus.cc:2379] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 5: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:53.374693 10929 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
from {username='slave'} at 127.0.0.1:54216
I20260809 15:52:53.374933 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.375247 10929 raft_consensus.cc:595] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Received request to transfer leadership
W20260809 15:52:53.377060 10789 auto_rebalancer.cc:663] Could not move replica: Remote error: Service unavailable: leadership transfer for 1f9b7d9e57934596b377ee8dc79ea23e already in progress
W20260809 15:52:53.377427 10789 auto_rebalancer.cc:264] scheduled replica move failed to complete: Remote error: Service unavailable: leadership transfer for 1f9b7d9e57934596b377ee8dc79ea23e already in progress
I20260809 15:52:53.378922 10844 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:52:53.382400 13913 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:53.306035 14172 consensus_queue.cc:260] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 4.4, Last appended by leader: 4, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:53.384822 14172 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
W20260809 15:52:53.386181 14188 consensus_queue.cc:1159] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "a53441872c0c416b8e2f26f52399f536" responder_term: 4 status { last_received { term: 4 index: 4 } last_committed_idx: 4 last_received_current_leader { term: 4 index: 4 } }
I20260809 15:52:53.350481 13999 consensus_queue.cc:1035] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260809 15:52:53.388652 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:53.344386 11018 rpcz_store.cc:276] 0809 15:52:52.315245 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:52.832223 (+516978us) service_pool.cc:227] Handling call
0809 15:52:52.832580 (+ 357us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:52.832862 (+ 282us) raft_consensus.cc:1520] Early marking committed up to index 6
0809 15:52:53.052210 (+219348us) log.cc:842] Serialized 14 byte log entry
0809 15:52:53.343613 (+291403us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:53.343669 (+ 56us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:52:53.343708 (+ 39us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:53.343769 (+ 61us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:53.343869 (+ 100us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":4105,"prepare.queue_time_us":180520,"prepare.run_cpu_time_us":25,"prepare.run_wall_time_us":26,"raft.queue_time_us":18899,"raft.run_cpu_time_us":65,"raft.run_wall_time_us":64,"thread_start_us":438821,"threads_started":3,"wal-append.queue_time_us":279365}
I20260809 15:52:53.393085 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.392381 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.394366 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.394889 13979 consensus_queue.cc:1035] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 15:52:53.397536 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.398355 10927 raft_consensus.cc:2379] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 6: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:53.399153 13958 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:53.399977 13906 raft_consensus.cc:2726] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:53.401077 13999 consensus_queue.cc:1035] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:53.399089 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.394706 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:53.412055 11015 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 7.
I20260809 15:52:53.423959 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.424669 10922 raft_consensus.cc:2454] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:53.424861 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:53.426666 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:53.427205 11015 raft_consensus.cc:3037] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:53.430419 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:53.433380 11010 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:53.435214 11015 raft_consensus.cc:2454] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
W20260809 15:52:53.438264 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 147562us
I20260809 15:52:53.442812 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:53.443665 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:53.445907 10847 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:53.425818 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.446565 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:53.457939 10978 connection.cc:232] RPC call timeout handler was delayed by -1.6990324530052021s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
W20260809 15:52:53.458556 10978 connection.cc:232] RPC call timeout handler was delayed by -1.636032453039661s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1888s for a response
W20260809 15:52:53.458945 10978 connection.cc:232] RPC call timeout handler was delayed by -1.636032453039661s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1888s for a response
W20260809 15:52:53.459321 10978 connection.cc:232] RPC call timeout handler was delayed by -1.5973324530059472s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
W20260809 15:52:53.459717 10978 connection.cc:232] RPC call timeout handler was delayed by -1.5973324530059472s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
W20260809 15:52:53.460078 10978 connection.cc:232] RPC call timeout handler was delayed by -1.5973324530059472s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.19310000000000002s for a response
W20260809 15:52:53.460423 10978 connection.cc:232] RPC call timeout handler was delayed by -1.5937324530095793s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1935s for a response
W20260809 15:52:53.460763 10978 connection.cc:232] RPC call timeout handler was delayed by -1.5937324530095793s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1935s for a response
W20260809 15:52:53.461436 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 98235.8us
I20260809 15:52:53.464202 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.450922 10809 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:53.474651 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:53.480293 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:53.480959 11010 raft_consensus.cc:1226] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:53.482115 11015 raft_consensus.cc:2379] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 5: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:53.482440 11606 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:53.480846 10809 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:53.444661 13958 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:53.498943 11604 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:53.499907 10931 raft_consensus.cc:1226] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:53.484800 14033 raft_consensus.cc:2781] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:53.491954 14094 raft_consensus.cc:2781] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:53.454630 13798 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.049s
I20260809 15:52:53.505630 14047 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:53.506984 10922 raft_consensus.cc:1226] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: [3.6-3.6]
I20260809 15:52:53.508879 14005 raft_consensus.cc:2781] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:52:53.509374 14167 consensus_queue.cc:1046] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 15:52:53.510308 14031 consensus_queue.cc:1035] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:53.511340 11010 raft_consensus.cc:3032] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 7 LEADER]: Stepping down as leader of term 7
I20260809 15:52:53.529765 11010 raft_consensus.cc:729] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:53.503592 14033 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:53.519181 10893 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:53.499716 14174 raft_consensus.cc:2726] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:53.529520 14094 raft_consensus.cc:686] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:53.535382 14094 consensus_queue.cc:237] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.537273 10893 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:53.539928 14031 raft_consensus.cc:2726] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:52:53.540342 10893 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:53.541081 14047 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:53.532876 14033 raft_consensus.cc:3037] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:53.500988 13958 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.546660 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:53.547497 11015 raft_consensus.cc:1226] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:53.548640 13958 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:53.549268 10978 leader_election.cc:334] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.633s (SENT)
I20260809 15:52:53.538846 14005 raft_consensus.cc:686] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 7 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:53.549721 10978 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:53.544808 13806 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:53.549844 11010 consensus_queue.cc:260] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.550868 10838 raft_consensus.cc:2379] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 6: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:53.550601 14005 consensus_queue.cc:237] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.544207 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:53.550961 11010 raft_consensus.cc:3037] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:53.544997 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.560731 10922 raft_consensus.cc:1226] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:53.565101 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:53.565691 11013 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:53.566916 10931 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:53.567166 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:53.567363 11018 pending_rounds.cc:76] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa: Aborting all ops after (but not including) 5
I20260809 15:52:53.567495 10931 raft_consensus.cc:595] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:53.567719 11018 pending_rounds.cc:97] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa: Aborting uncommitted NO_OP operation due to leader change: 6.6
I20260809 15:52:53.568038 11018 raft_consensus.cc:2866] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:53.568547 11017 raft_consensus.cc:2379] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 7: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:53.566931 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:53.569487 11014 raft_consensus.cc:3037] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:53.569371 14031 consensus_queue.cc:1035] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s
I20260809 15:52:53.570832 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:53.571444 11011 raft_consensus.cc:2454] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:53.567970 11015 raft_consensus.cc:1226] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:53.557248 13979 raft_consensus.cc:2781] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:53.575699 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.557991 13971 raft_consensus.cc:3032] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:53.557575 13798 raft_consensus.cc:2726] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:53.577593 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.577692 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.560473 14194 raft_consensus.cc:2726] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 15:52:53.578603 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.581121 10893 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:53.579631 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.586877 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.587244 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:53.589818 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:53.575525 13979 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:53.591675 13979 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:53.578567 13971 raft_consensus.cc:729] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:53.595297 13971 consensus_queue.cc:260] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.596495 13971 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:53.596644 14188 raft_consensus.cc:2781] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:53.597093 14188 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:53.597470 14188 raft_consensus.cc:3037] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:53.599005 11009 raft_consensus.cc:1226] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: [5.3-5.3]
I20260809 15:52:53.600667 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:53.579046 11606 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:53.603842 10893 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:53.625288 10978 connection.cc:232] RPC call timeout handler was delayed by -1.4956324530066922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20440000000000003s for a response
W20260809 15:52:53.625772 10978 connection.cc:232] RPC call timeout handler was delayed by -1.4956324530066922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20440000000000003s for a response
W20260809 15:52:53.626140 10978 connection.cc:232] RPC call timeout handler was delayed by -1.4956324530066922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20440000000000003s for a response
W20260809 15:52:53.626461 10978 connection.cc:232] RPC call timeout handler was delayed by -1.4956324530066922s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20440000000000003s for a response
W20260809 15:52:53.626833 10978 connection.cc:232] RPC call timeout handler was delayed by -1.4038324530120008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2146s for a response
W20260809 15:52:53.627178 10978 connection.cc:232] RPC call timeout handler was delayed by -1.4038324530120008s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2146s for a response
I20260809 15:52:53.627058 14033 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:53.627766 10978 connection.cc:232] RPC call timeout handler was delayed by -1.319232453010045s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22400000000000003s for a response
W20260809 15:52:53.628196 10978 connection.cc:232] RPC call timeout handler was delayed by -1.319232453010045s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22400000000000003s for a response
I20260809 15:52:53.628842 14047 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:53.629280 14033 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:53.629305 14047 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:53.630357 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:53.632133 10978 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:53.633301 10978 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:53.637538 14095 raft_consensus.cc:2781] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:53.638083 14095 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:53.638545 11011 raft_consensus.cc:1261] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260809 15:52:53.638595 14095 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:53.640492 14174 consensus_queue.cc:1035] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:53.634889 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:53.642338 13806 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:53.646370 10923 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:52:53.646754 10893 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:53.646971 10923 raft_consensus.cc:595] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:52:53.647945 10893 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:53.648530 13979 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:53.649277 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 70311.3us
I20260809 15:52:53.651011 13982 raft_consensus.cc:2726] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 15:52:53.651708 13982 raft_consensus.cc:2781] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:53.652045 13982 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:53.652400 13982 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:53.656100 10811 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:53.658420 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:53.570425 14172 consensus_queue.cc:1046] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.068s
I20260809 15:52:53.659348 13656 consensus_queue.cc:1046] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260809 15:52:53.659832 14048 raft_consensus.cc:2781] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:52:53.660296 14048 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:53.660521 13656 raft_consensus.cc:3032] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 15:52:53.660662 14048 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:53.660985 13656 raft_consensus.cc:729] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:53.662151 14030 raft_consensus.cc:2781] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 15:52:53.662536 14030 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:53.662878 14030 raft_consensus.cc:3037] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 15:52:53.661917 13656 consensus_queue.cc:260] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:53.668195 10811 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:53.678375 10837 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:53.665369 13656 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:53.684670 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.682960 10811 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:53.684254 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:53.687208 10837 raft_consensus.cc:1261] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 5. (term mismatch)
I20260809 15:52:53.687768 10837 pending_rounds.cc:76] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536: Aborting all ops after (but not including) 4
I20260809 15:52:53.688112 10837 pending_rounds.cc:97] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260809 15:52:53.688480 10837 raft_consensus.cc:2866] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:53.684602 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:53.700067 10838 raft_consensus.cc:2396] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260809 15:52:53.701309 10811 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:53.702023 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:53.704946 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:53.705564 10842 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 7
I20260809 15:52:53.705848 10844 raft_consensus.cc:1226] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:53.706213 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:53.706830 10840 raft_consensus.cc:3037] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:53.707068 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.709226 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:53.709766 10848 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:53.717443 11010 raft_consensus.cc:1261] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260809 15:52:53.717902 11010 pending_rounds.cc:76] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa: Aborting all ops after (but not including) 8
I20260809 15:52:53.718264 11010 pending_rounds.cc:97] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa: Aborting uncommitted NO_OP operation due to leader change: 7.9
I20260809 15:52:53.718899 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:53.719537 10841 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:53.720101 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:53.725169 11016 raft_consensus.cc:1261] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260809 15:52:53.720885 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:53.659832 14171 consensus_queue.cc:1035] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.045s
I20260809 15:52:53.733788 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:53.734372 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:53.734889 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:53.734925 10838 raft_consensus.cc:3037] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:53.736454 11015 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:53.737186 11010 raft_consensus.cc:2866] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:53.684701 13999 raft_consensus.cc:2726] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 15:52:53.738917 11639 raft_consensus.cc:2379] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 5: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:53.757598 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:53.758139 10811 leader_election.cc:302] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:53.758206 10847 raft_consensus.cc:1226] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:53.708833 13973 raft_consensus.cc:2781] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:53.768622 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:53.772008 11641 raft_consensus.cc:1799] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Handling vote request from an unknown peer d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:53.772459 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:53.769366 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:53.699831 14094 raft_consensus.cc:2726] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:53.775276 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.770607 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:53.776724 11641 raft_consensus.cc:2396] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for term 8 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260809 15:52:53.777483 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:53.779021 10931 raft_consensus.cc:2379] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 7: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:53.779564 11636 raft_consensus.cc:2454] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:53.776235 10809 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:53.784608 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:53.783792 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:53.718675 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:53.725517 14133 raft_consensus.cc:2726] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:53.781930 10891 leader_election.cc:302] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:53.792366 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 144522us
I20260809 15:52:53.789912 10842 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 7.
I20260809 15:52:53.767282 13979 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:53.773017 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:53.774049 13973 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:53.848518 13973 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:53.718950 14031 consensus_queue.cc:1035] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.070s
I20260809 15:52:53.775549 14163 raft_consensus.cc:2726] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:53.795320 14133 raft_consensus.cc:2781] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:53.844172 10978 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:53.780083 14123 consensus_queue.cc:1035] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.067s
I20260809 15:52:53.795766 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:53.799032 10930 raft_consensus.cc:2379] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 8: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:53.778913 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:53.846503 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:53.841923 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
W20260809 15:52:53.852310 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 70419us
I20260809 15:52:53.853803 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:53.854723 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:53.903505 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 59418.7us
I20260809 15:52:53.855059 10931 raft_consensus.cc:1226] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260809 15:52:53.856195 13798 raft_consensus.cc:2781] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:53.775537 13913 consensus_queue.cc:1035] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.070s
W20260809 15:52:53.843506 14208 log.cc:924] Time spent T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.139s user 0.000s sys 0.001s
I20260809 15:52:53.862499 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:53.857905 11014 raft_consensus.cc:2454] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:53.872083 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:53.901135 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.914889 11639 raft_consensus.cc:2379] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 5: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:53.851946 14047 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:53.915850 10929 raft_consensus.cc:1226] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:53.900226 11013 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:53.782575 14033 consensus_queue.cc:1046] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
W20260809 15:52:53.902973 14166 log.cc:924] Time spent T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.059s user 0.000s sys 0.000s
I20260809 15:52:53.918603 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:53.921718 10931 raft_consensus.cc:1226] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260809 15:52:53.923983 11606 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:53.922708 11639 raft_consensus.cc:2379] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 5: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:53.921784 10923 raft_consensus.cc:1261] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260809 15:52:53.905561 13798 raft_consensus.cc:482] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:53.909962 13917 raft_consensus.cc:2781] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election won for term 6
W20260809 15:52:53.922117 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 151486us
I20260809 15:52:53.910504 14133 raft_consensus.cc:686] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:53.929242 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:53.930318 10811 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:53.931668 14048 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:53.932899 10811 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:53.863548 14095 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:53.918754 14195 raft_consensus.cc:2741] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260809 15:52:53.930624 11636 raft_consensus.cc:2379] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 8: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:53.940900 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:53.929572 13798 raft_consensus.cc:3037] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:53.960320 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:53.961964 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:53.968240 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.968729 11010 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:53.921394 13893 consensus_queue.cc:1035] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.035s
I20260809 15:52:53.970999 10978 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:53.971134 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:54.043242 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:54.044654 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:54.041000 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:54.044282 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:54.048168 11636 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 8.
I20260809 15:52:54.051254 10841 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 8.
I20260809 15:52:53.960134 13917 raft_consensus.cc:686] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:54.056306 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:54.060916 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:54.061010 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:54.061525 11639 raft_consensus.cc:2379] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 4: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:54.061695 11636 raft_consensus.cc:2379] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 6: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:54.043893 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:54.063031 10924 raft_consensus.cc:1226] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 5. Current term is 6. Ops: [5.4-5.4]
I20260809 15:52:54.058533 10923 raft_consensus.cc:2454] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:54.061717 10929 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:52:53.978837 11606 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
W20260809 15:52:54.042001 14207 log.cc:924] Time spent T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.172s user 0.000s sys 0.003s
I20260809 15:52:54.064234 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:54.070326 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.974152 13806 raft_consensus.cc:2726] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:52:54.043402 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:54.073185 11010 raft_consensus.cc:2396] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 8 because replica has last-logged OpId of term: 6 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 4.
I20260809 15:52:54.082881 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.959941 14048 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:54.042663 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:54.037387 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:54.081248 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:54.074747 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:54.079722 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:53.955679 14004 consensus_queue.cc:1046] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.103s
I20260809 15:52:54.083633 10923 raft_consensus.cc:2379] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 5: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:54.079893 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
W20260809 15:52:54.084039 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 105264us
I20260809 15:52:54.084686 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.959002 13906 raft_consensus.cc:2726] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:54.090395 14216 consensus_queue.cc:1046] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.121s
I20260809 15:52:54.071542 14123 consensus_queue.cc:1035] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.030s
I20260809 15:52:54.096484 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:54.150425 10926 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:53.955829 14217 consensus_queue.cc:1046] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.146s
I20260809 15:52:54.101593 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:53.972355 14164 raft_consensus.cc:2726] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:54.139271 11642 raft_consensus.cc:2379] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 4: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:54.097702 14192 raft_consensus.cc:2781] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:52:54.142215 14048 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:54.158437 10920 raft_consensus.cc:2379] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 3: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:54.156886 14194 raft_consensus.cc:2741] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election decision vote started in defunct term 2: won
W20260809 15:52:54.141598 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 181034us
I20260809 15:52:54.145967 10931 raft_consensus.cc:2379] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 7: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:54.152518 11606 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:54.156090 10838 raft_consensus.cc:2454] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:54.182641 11642 raft_consensus.cc:1226] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260809 15:52:54.190152 11606 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
W20260809 15:52:54.178970 13982 consensus_meta.cc:287] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.526s user 0.006s sys 0.006s
I20260809 15:52:54.140300 11636 raft_consensus.cc:2454] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:54.191653 11606 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:54.185833 14048 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:54.200331 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:54.100720 14133 consensus_queue.cc:237] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:54.180066 14192 raft_consensus.cc:686] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 7 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
W20260809 15:52:54.224311 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 288975us
I20260809 15:52:54.225279 11606 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:54.100216 13917 consensus_queue.cc:237] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:54.225345 10891 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:54.230783 10891 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:54.202613 13893 raft_consensus.cc:2781] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 15:52:54.234385 11606 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:54.185030 13973 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:54.203619 13906 raft_consensus.cc:2726] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:54.237110 11011 raft_consensus.cc:1261] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260809 15:52:54.243902 14213 raft_consensus.cc:3032] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:54.244261 14213 raft_consensus.cc:729] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:54.245007 14213 consensus_queue.cc:260] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.4, Last appended by leader: 4, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:54.248219 14220 raft_consensus.cc:2726] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260809 15:52:54.249620 10840 consensus_meta.cc:287] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.511s user 0.003s sys 0.004s
I20260809 15:52:54.234936 10848 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:54.249938 10840 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:54.259824 10978 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:54.190918 13979 consensus_queue.cc:1035] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.044s
I20260809 15:52:54.232317 14171 raft_consensus.cc:2726] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 15:52:54.262838 11637 raft_consensus.cc:2454] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
I20260809 15:52:54.271543 10839 raft_consensus.cc:1226] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260809 15:52:54.237367 14095 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:54.205179 13982 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:54.260685 14169 raft_consensus.cc:2726] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
I20260809 15:52:54.295710 10811 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:54.295701 10978 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
W20260809 15:52:54.296855 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 237121us
I20260809 15:52:54.299652 10840 raft_consensus.cc:1226] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:54.275458 13893 raft_consensus.cc:686] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 8 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:54.308087 10927 raft_consensus.cc:2379] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 7: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:54.313205 10930 raft_consensus.cc:1226] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:54.313531 10929 raft_consensus.cc:1261] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260809 15:52:54.305219 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 101824us
I20260809 15:52:54.327957 10811 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:54.294235 14245 raft_consensus.cc:979] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:54.329360 10978 leader_election.cc:302] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:54.331424 10931 raft_consensus.cc:3032] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:54.310690 14194 raft_consensus.cc:2726] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:54.292425 14171 consensus_queue.cc:1035] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.026s
I20260809 15:52:54.334014 10931 raft_consensus.cc:729] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:54.312799 14123 raft_consensus.cc:2726] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:52:54.319306 14216 raft_consensus.cc:3032] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:54.341017 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:54.341171 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:54.338256 14095 raft_consensus.cc:2781] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260809 15:52:54.302737 14167 consensus_queue.cc:1046] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.032s
I20260809 15:52:54.352008 10811 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
W20260809 15:52:54.334774 14030 consensus_meta.cc:287] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.671s user 0.011s sys 0.004s
I20260809 15:52:54.357273 11606 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:54.356437 10893 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
W20260809 15:52:54.340173 14212 log.cc:924] Time spent T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.349s user 0.001s sys 0.004s
W20260809 15:52:54.355098 14243 log.cc:924] Time spent T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.063s user 0.000s sys 0.000s
I20260809 15:52:54.360195 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:54.363488 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:54.291877 14220 consensus_queue.cc:1046] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260809 15:52:54.357532 10931 consensus_queue.cc:260] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:54.339181 14228 raft_consensus.cc:2781] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:54.376942 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:54.378137 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:54.356336 14256 raft_consensus.cc:1067] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:52:54.383620 10893 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:54.385201 10978 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:54.378957 11642 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 10.
I20260809 15:52:54.341950 14216 raft_consensus.cc:729] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:54.390062 10811 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:54.359639 14095 raft_consensus.cc:482] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:54.354472 14238 consensus_queue.cc:1035] T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260809 15:52:54.393765 10978 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:54.392087 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:54.391227 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 160493us
I20260809 15:52:54.396571 11642 raft_consensus.cc:2396] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 6 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260809 15:52:54.307694 14192 consensus_queue.cc:237] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:54.387871 14233 log.cc:924] Time spent T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.052s user 0.000s sys 0.001s
I20260809 15:52:54.383353 14167 consensus_queue.cc:1225] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer 20730d75ad954a82b126148f8152a9f3 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260809 15:52:54.403445 11636 raft_consensus.cc:1261] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260809 15:52:54.407191 11016 raft_consensus.cc:1226] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4. Ops: [3.4-3.4]
I20260809 15:52:54.387681 14123 raft_consensus.cc:2726] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260809 15:52:54.407713 10978 raft_consensus.cc:2670] T 5e8d12b5ef9e415cbcb25e62ab228546 P d74944fa9fa14ec78b0640ea709ff3aa: unable to run election callback: Service unavailable: Thread pool token was shut down
I20260809 15:52:54.408479 10978 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:54.417717 10931 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:54.420778 11015 raft_consensus.cc:1226] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:54.416569 10926 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:54.391731 13979 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:54.389848 14194 raft_consensus.cc:2781] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:54.396157 14095 raft_consensus.cc:3037] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:54.423868 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:54.390612 14249 raft_consensus.cc:2726] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 15:52:54.401081 14215 raft_consensus.cc:2781] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:54.394424 14228 raft_consensus.cc:686] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 6 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:54.427234 11016 raft_consensus.cc:2379] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 5: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:54.391533 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:54.374197 14030 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:54.428188 14248 raft_consensus.cc:2781] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:52:54.428467 14123 raft_consensus.cc:2781] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 15:52:54.428588 14247 raft_consensus.cc:2726] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 15:52:54.424700 14194 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:54.433629 11639 raft_consensus.cc:2454] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:54.422115 14048 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:54.418720 13973 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:54.467329 14030 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:54.461014 14231 consensus_queue.cc:1046] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.079s
I20260809 15:52:54.436605 14215 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:54.477777 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:54.478991 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:54.481587 10926 raft_consensus.cc:2379] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 6: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:54.401201 14229 consensus_queue.cc:1046] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.181s
I20260809 15:52:54.447024 13982 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:54.485399 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:54.487496 11016 raft_consensus.cc:2354] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3.
I20260809 15:52:54.486135 10931 raft_consensus.cc:1261] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260809 15:52:54.453879 14123 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:54.491313 10924 raft_consensus.cc:1261] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 4. Preceding OpId from leader: term: 5 index: 5. (index mismatch)
I20260809 15:52:54.459007 14194 raft_consensus.cc:3037] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:54.395589 13893 consensus_queue.cc:237] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:54.494673 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:54.495976 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:54.499866 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 109870us
W20260809 15:52:54.485641 14188 consensus_meta.cc:287] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.888s user 0.006s sys 0.006s
W20260809 15:52:54.507819 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 99427.1us
I20260809 15:52:54.452600 11632 raft_consensus.cc:2362] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7. Re-sending same reply.
I20260809 15:52:54.506553 10811 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:54.511656 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:54.502317 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:54.512243 10926 raft_consensus.cc:3037] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:54.513820 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:54.508782 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 104335us
W20260809 15:52:54.507144 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 152620us
I20260809 15:52:54.457739 14248 raft_consensus.cc:686] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:54.484186 14215 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:54.502805 14123 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Advancing to term 9
W20260809 15:52:54.508385 14236 log.cc:924] Time spent T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.205s user 0.001s sys 0.000s
I20260809 15:52:54.522473 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:54.534744 10840 raft_consensus.cc:2379] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:54.512574 14188 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:54.523360 11634 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:54.518777 11606 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:54.501385 14229 raft_consensus.cc:2726] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 15:52:54.530272 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:54.530704 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:54.544106 10838 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:54.537655 10811 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
W20260809 15:52:54.519511 13656 consensus_meta.cc:287] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.820s user 0.014s sys 0.001s
I20260809 15:52:54.532300 11009 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50542
I20260809 15:52:54.541610 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:54.454259 14216 consensus_queue.cc:260] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:54.543696 10926 raft_consensus.cc:2454] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 8.
I20260809 15:52:54.525826 13982 consensus_queue.cc:1046] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 1.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260809 15:52:54.557993 10893 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:54.559192 10893 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:54.561604 10893 leader_election.cc:302] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:54.562610 14255 log.cc:924] Time spent T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.088s user 0.001s sys 0.000s
I20260809 15:52:54.563736 14261 raft_consensus.cc:2726] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:54.568883 13982 raft_consensus.cc:2726] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:52:54.569707 13656 raft_consensus.cc:2726] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:54.573448 10922 raft_consensus.cc:2396] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 5 because replica has last-logged OpId of term: 3 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260809 15:52:54.552160 11009 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:52:54.574462 11009 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:54.552709 10844 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 8.
I20260809 15:52:54.545625 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:54.575392 10840 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:52:54.525696 14249 consensus_queue.cc:1046] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.029s
W20260809 15:52:54.577572 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 55603.6us
W20260809 15:52:54.582710 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 45155.8us
I20260809 15:52:54.583506 10811 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:54.598949 10838 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:54.602905 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:54.603595 11640 raft_consensus.cc:2362] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6. Re-sending same reply.
I20260809 15:52:54.577171 13982 raft_consensus.cc:2726] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:54.604658 13656 raft_consensus.cc:3037] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:54.630890 10811 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:54.631979 10811 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:54.531368 14228 consensus_queue.cc:237] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:54.655923 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 114403us
W20260809 15:52:54.656550 10978 connection.cc:232] RPC call timeout handler was delayed by -1.0083949799882248s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2117s for a response
W20260809 15:52:54.657137 10978 connection.cc:232] RPC call timeout handler was delayed by -1.0083949799882248s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2117s for a response
I20260809 15:52:54.625141 11009 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:54.627800 14217 raft_consensus.cc:2781] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:54.666254 11009 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:54.654161 13825 raft_consensus.cc:2781] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:52:54.653990 14261 raft_consensus.cc:2781] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election won for term 5
W20260809 15:52:54.694223 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 112838us
W20260809 15:52:54.708778 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 122283us
I20260809 15:52:54.618673 14249 consensus_queue.cc:1035] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.050s
I20260809 15:52:54.710089 14261 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:54.738306 11013 raft_consensus.cc:1261] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260809 15:52:54.731784 13973 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:54.617272 14248 consensus_queue.cc:237] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:54.747545 13798 consensus_meta.cc:287] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.689s user 0.007s sys 0.005s
I20260809 15:52:54.752139 13973 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:54.740163 14188 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:54.771126 14265 log.cc:924] Time spent T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.162s user 0.001s sys 0.000s
I20260809 15:52:54.739753 14261 raft_consensus.cc:3037] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:54.766971 10978 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:54.775915 10978 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:54.777758 10978 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:54.786664 10978 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
W20260809 15:52:54.791776 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 73723.8us
I20260809 15:52:54.781839 14229 raft_consensus.cc:2781] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:54.782281 13917 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260809 15:52:54.760969 14247 consensus_queue.cc:1035] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.030s
I20260809 15:52:54.773128 13798 raft_consensus.cc:504] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:54.812487 14281 consensus_queue.cc:1035] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.007s
I20260809 15:52:54.813719 13973 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:54.814254 14264 raft_consensus.cc:2726] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:54.815308 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:54.804725 14229 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:54.806752 13917 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:54.821197 10921 raft_consensus.cc:2379] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 3: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:54.822835 10920 raft_consensus.cc:1226] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:54.820864 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
W20260809 15:52:54.821177 14277 log.cc:924] Time spent T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.070s user 0.000s sys 0.001s
I20260809 15:52:54.811968 14047 raft_consensus.cc:2726] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:54.825059 13917 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Advancing to term 11
I20260809 15:52:54.836256 13798 leader_election.cc:288] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:54.820968 14229 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:54.836321 14123 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:54.837157 14274 log.cc:924] Time spent T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.051s user 0.000s sys 0.001s
I20260809 15:52:54.838809 14217 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:54.838963 13656 raft_consensus.cc:2726] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260809 15:52:54.839707 14226 log.cc:924] Time spent T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.614s user 0.003s sys 0.002s
I20260809 15:52:54.839917 14261 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:54.842528 14194 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:54.843911 14237 raft_consensus.cc:3032] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:54.844094 13979 raft_consensus.cc:2726] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260809 15:52:54.850772 14206 log.cc:924] Time spent T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.944s user 0.001s sys 0.003s
I20260809 15:52:54.852692 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:54.865532 14047 raft_consensus.cc:3032] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:54.879211 13825 raft_consensus.cc:686] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 7 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:54.880229 11009 raft_consensus.cc:1226] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
W20260809 15:52:54.960469 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 76073.1us
W20260809 15:52:54.880431 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 93876.5us
W20260809 15:52:54.881436 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 141445us
I20260809 15:52:54.881937 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:54.932194 11637 raft_consensus.cc:2379] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 4: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:54.962180 10978 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:54.886488 14217 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:54.965054 11013 raft_consensus.cc:1226] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:55.001058 10978 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:55.005498 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 204581us
I20260809 15:52:55.006484 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:55.006831 10811 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:55.007262 10838 raft_consensus.cc:2354] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8.
I20260809 15:52:54.930531 14237 raft_consensus.cc:729] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:55.007705 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:55.008424 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:55.009770 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:55.009915 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:55.011798 10811 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:55.012364 10848 raft_consensus.cc:2396] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 6 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 3.
I20260809 15:52:54.960625 14047 raft_consensus.cc:729] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:55.015556 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:55.014268 14206 log.cc:924] Time spent T 279308d651cf45ff86edaae4483bcb79 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.056s user 0.000s sys 0.000s
I20260809 15:52:55.028476 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:55.022367 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:55.030272 10923 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:55.027518 14194 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:55.031692 14194 raft_consensus.cc:2781] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:52:55.032068 14123 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:55.015358 14300 raft_consensus.cc:2781] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 15:52:55.033067 14300 raft_consensus.cc:686] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 8 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:55.029007 13798 consensus_queue.cc:1046] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 4.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.006s
I20260809 15:52:55.035355 10931 raft_consensus.cc:1226] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:55.036252 14296 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:55.026098 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:55.026736 14289 log.cc:924] Time spent T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.065s user 0.000s sys 0.001s
W20260809 15:52:55.029488 14095 consensus_meta.cc:287] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.573s user 0.012s sys 0.005s
I20260809 15:52:55.039526 14095 raft_consensus.cc:504] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.022943 14237 consensus_queue.cc:260] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.041553 14095 leader_election.cc:288] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:54.885927 14290 raft_consensus.cc:2932] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Committing config change with OpId 5.6: config changed from index 4 to 6, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:55.029724 10926 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:52:55.042470 11014 raft_consensus.cc:2379] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 3: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:55.033977 14300 consensus_queue.cc:237] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:55.021184 13973 consensus_queue.cc:1046] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.025s
I20260809 15:52:55.046247 14237 raft_consensus.cc:3037] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
W20260809 15:52:55.047942 14215 consensus_meta.cc:287] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.508s user 0.008s sys 0.002s
I20260809 15:52:55.026386 13825 consensus_queue.cc:237] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.048468 14215 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.032094 14194 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:55.050719 14215 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:55.056485 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:55.060413 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:55.061417 10838 raft_consensus.cc:2379] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 6: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:55.051209 14194 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:55.016479 14293 raft_consensus.cc:2726] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:55.073261 10837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:45572 (request call id 983) took 1280 ms. Trace:
I20260809 15:52:55.056707 13656 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:55.076298 10837 rpcz_store.cc:276] 0809 15:52:53.793221 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:53.793367 (+ 146us) service_pool.cc:227] Handling call
0809 15:52:55.072400 (+1279033us) spinlock_profiling.cc:243] Waited 3.27 ms on lock 0x7b5800004298. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f7966593851 00007f7966593bce 000000000034aea9 00007f796e3bfa30 00007f796e485741 00007f796e485900 00007f796e438b50 00007f7967b2f807 00007f7967b2f792 00007f7967b2f6c1
0809 15:52:55.073171 (+ 771us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":7530368}
I20260809 15:52:55.076095 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:55.076241 13906 raft_consensus.cc:3032] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:55.080403 13906 raft_consensus.cc:729] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
W20260809 15:52:55.083096 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 82132us
I20260809 15:52:55.084481 10978 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:55.086198 10978 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:55.088805 14215 raft_consensus.cc:2781] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:55.089020 10978 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:55.089239 14215 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:55.089633 14215 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:55.089888 14306 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:55.090547 10978 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:55.091457 10978 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:55.093310 10839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 781) took 1412 ms. Trace:
I20260809 15:52:55.082970 13656 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:55.093712 10839 rpcz_store.cc:276] 0809 15:52:53.676287 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:53.676966 (+ 679us) service_pool.cc:227] Handling call
0809 15:52:53.720825 (+ 43859us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:54.271943 (+551118us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:55.088103 (+816160us) spinlock_profiling.cc:243] Waited 548 ms on lock 0x7b54003ae654. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:55.088312 (+ 209us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1260494208}
I20260809 15:52:55.091272 14095 raft_consensus.cc:2741] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260809 15:52:55.028004 14047 consensus_queue.cc:260] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.098485 10837 raft_consensus.cc:1226] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:55.081245 13906 consensus_queue.cc:260] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.092269 14293 raft_consensus.cc:2726] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:55.104498 13656 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:55.105114 14297 raft_consensus.cc:2726] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260809 15:52:55.105710 10922 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52758 (request call id 796) took 1433 ms (client timeout 1873 ms). Trace:
W20260809 15:52:55.106172 10922 rpcz_store.cc:269] 0809 15:52:53.672598 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:53.672744 (+ 146us) service_pool.cc:227] Handling call
0809 15:52:55.105511 (+1432767us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b540025bcd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:55.105655 (+ 144us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1990554496}
I20260809 15:52:55.109128 10923 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:52:55.119813 10978 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:55.128594 10978 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:55.130131 14297 raft_consensus.cc:2781] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:55.130645 14297 raft_consensus.cc:686] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:55.131497 14297 consensus_queue.cc:237] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:55.133533 14195 raft_consensus.cc:2726] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:55.133909 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:55.106366 13906 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:55.120499 14047 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:55.146729 10978 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:55.148494 11633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41482 (request call id 704) took 1410 ms (client timeout 1655 ms). Trace:
I20260809 15:52:55.148782 10839 raft_consensus.cc:1226] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 8. Current term is 9. Ops: []
I20260809 15:52:55.157632 10848 raft_consensus.cc:1226] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260809 15:52:55.157935 14223 raft_consensus.cc:3032] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:55.161549 10978 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:55.159341 10841 raft_consensus.cc:1226] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: [3.2-5.3]
I20260809 15:52:55.121531 14261 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:55.168766 14223 raft_consensus.cc:729] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:55.182509 14095 raft_consensus.cc:2781] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:55.183101 14293 raft_consensus.cc:2781] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:52:55.185935 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:52:55.168728 11633 rpcz_store.cc:269] 0809 15:52:53.738030 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:53.762499 (+ 24469us) service_pool.cc:227] Handling call
0809 15:52:55.133753 (+1371254us) spinlock_profiling.cc:243] Waited 1.16 ms on lock 0x7b5800014198. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f7966593851 00007f7966593702 00007f796f483b09 00007f796e3bf730 00007f796e48579b 00007f796e485821 00007f796e4351b0 00007f796e434a94 00007f7967b1cc04 00007f7967b1cb92
0809 15:52:55.148433 (+ 14680us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2668928}
I20260809 15:52:55.186741 10848 raft_consensus.cc:1226] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260809 15:52:55.191906 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:55.194455 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:55.209496 11009 raft_consensus.cc:1226] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:55.211170 10922 raft_consensus.cc:3032] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:55.214540 10922 raft_consensus.cc:729] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:55.216213 10922 consensus_queue.cc:260] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.221261 10922 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:55.258137 10921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 2054) took 1393 ms. Trace:
I20260809 15:52:55.258602 10921 rpcz_store.cc:276] 0809 15:52:53.864654 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:53.936132 (+ 71478us) service_pool.cc:227] Handling call
0809 15:52:55.257856 (+1321724us) spinlock_profiling.cc:243] Waited 852 ms on lock 0x7b540030acd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:55.258045 (+ 189us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1959667072}
I20260809 15:52:55.231017 10893 leader_election.cc:398] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Vote denied by peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) with higher term. Message: Invalid argument: T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8.
I20260809 15:52:55.237645 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:55.203243 14293 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:55.210589 14261 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:55.230599 14296 raft_consensus.cc:2726] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:55.251917 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:55.255560 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:55.261960 10848 raft_consensus.cc:2379] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 5: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:55.259564 10893 leader_election.cc:401] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260809 15:52:55.187428 13917 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.277362 10839 raft_consensus.cc:2379] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:55.279711 10841 raft_consensus.cc:1226] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:55.238900 10847 raft_consensus.cc:1226] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4. Ops: []
W20260809 15:52:55.282581 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 46037.1us
I20260809 15:52:55.285236 10978 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:55.286736 10978 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:55.288110 10978 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:55.296360 14095 raft_consensus.cc:686] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
W20260809 15:52:55.297379 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 294035us
I20260809 15:52:55.302798 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:55.303646 10840 raft_consensus.cc:2379] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 7: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:55.304579 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:55.305727 14217 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.307991 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:55.308907 10931 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:55.308238 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:55.310127 10848 raft_consensus.cc:2379] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 4: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:55.310374 13917 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 11 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:55.310860 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:55.310875 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:55.311728 10848 raft_consensus.cc:2454] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:55.312381 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:55.312855 11010 raft_consensus.cc:1226] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:55.313086 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:55.314057 10848 raft_consensus.cc:1226] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:55.313808 14306 raft_consensus.cc:2726] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:55.310256 13825 consensus_queue.cc:1046] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:55.318473 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:55.313782 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:55.273209 14293 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:55.320746 10921 raft_consensus.cc:2454] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:55.302320 14297 raft_consensus.cc:2726] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260809 15:52:55.377921 14315 log.cc:924] Time spent T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.064s user 0.001s sys 0.000s
W20260809 15:52:55.374501 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 49589us
I20260809 15:52:55.325387 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:55.393232 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:55.289131 14195 raft_consensus.cc:2726] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:55.310962 13917 raft_consensus.cc:3037] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
W20260809 15:52:55.391079 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 185258us
I20260809 15:52:55.401494 10847 raft_consensus.cc:1261] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 15:52:55.404579 10922 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:55.379933 10848 raft_consensus.cc:1226] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:55.289395 14245 raft_consensus.cc:2726] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) with higher term. Message: Invalid argument: T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8.
I20260809 15:52:55.255015 14223 consensus_queue.cc:260] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.407721 10811 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:55.407837 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:55.290419 14261 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.392727 14300 raft_consensus.cc:3032] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:52:55.412251 10893 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:55.430518 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:55.431361 11010 raft_consensus.cc:1261] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:52:55.429647 10811 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:55.433584 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:55.435577 10926 raft_consensus.cc:2379] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 3: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:55.399232 14318 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:55.460112 14245 raft_consensus.cc:2726] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:55.460706 14318 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:55.463431 14216 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:55.431427 14123 raft_consensus.cc:2781] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:55.440330 14217 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:55.447242 14278 consensus_queue.cc:1035] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:55.443018 14095 consensus_queue.cc:237] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 3.2, Last appended by leader: 2, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:55.455119 14261 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:55.455346 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:55.458817 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 50772.2us
I20260809 15:52:55.478425 11010 raft_consensus.cc:2454] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:55.479545 10893 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:55.480495 10809 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:55.480893 10893 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:55.481338 14261 raft_consensus.cc:2781] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:55.481773 14261 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:55.470227 14123 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:55.482198 14261 raft_consensus.cc:3037] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:55.482388 14123 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:55.482707 10893 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:55.483879 14288 log.cc:924] Time spent T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.473s user 0.002s sys 0.000s
I20260809 15:52:55.484584 14245 raft_consensus.cc:2726] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:55.484783 13979 consensus_queue.cc:1046] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:55.485416 14245 consensus_queue.cc:1046] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 15:52:55.485996 13979 raft_consensus.cc:3032] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:55.486409 13979 raft_consensus.cc:729] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:55.487035 14287 raft_consensus.cc:3032] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 15:52:55.487536 14287 raft_consensus.cc:729] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:55.487263 13979 consensus_queue.cc:260] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.488673 14287 consensus_queue.cc:260] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.489943 14215 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:55.490217 14256 consensus_queue.cc:1046] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:55.491262 13999 raft_consensus.cc:3032] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:55.491605 13999 raft_consensus.cc:729] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:55.492017 14217 consensus_queue.cc:1046] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 15:52:55.492339 13999 consensus_queue.cc:260] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.492792 14305 consensus_queue.cc:1046] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 15:52:55.493597 13999 raft_consensus.cc:3037] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:55.493919 14319 consensus_queue.cc:1046] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 3.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260809 15:52:55.494585 14267 raft_consensus.cc:3032] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:55.495026 14267 raft_consensus.cc:729] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:55.495592 14312 raft_consensus.cc:3032] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 15:52:55.496025 14312 raft_consensus.cc:729] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:55.495951 14267 consensus_queue.cc:260] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:55.497143 11011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2136) took 1181 ms. Trace:
I20260809 15:52:55.497463 14267 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:55.498266 14285 raft_consensus.cc:3032] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Stepping down as leader of term 3
W20260809 15:52:55.498632 10844 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 793) took 1724 ms (client timeout 1548 ms). Trace:
I20260809 15:52:55.498759 14285 raft_consensus.cc:729] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:55.498974 14215 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:55.499104 10844 rpcz_store.cc:269] 0809 15:52:53.773869 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:53.792572 (+ 18703us) service_pool.cc:227] Handling call
0809 15:52:55.498445 (+1705873us) spinlock_profiling.cc:243] Waited 650 ms on lock 0x7b54000596d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:55.498543 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1494449024}
I20260809 15:52:55.499794 14285 consensus_queue.cc:260] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.484829 14252 raft_consensus.cc:2726] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 15:52:55.497624 14163 raft_consensus.cc:3032] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:52:55.501267 14163 raft_consensus.cc:729] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:55.493824 11014 raft_consensus.cc:1226] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:55.497587 11011 rpcz_store.cc:276] 0809 15:52:54.315523 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:54.328089 (+ 12566us) service_pool.cc:227] Handling call
0809 15:52:54.328360 (+ 271us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:54.430516 (+102156us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:54.430545 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:54.430588 (+ 43us) raft_consensus.cc:1631] Marking committed up to 8
0809 15:52:54.430615 (+ 27us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:55.496894 (+1066279us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b540041ffd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:55.496943 (+ 49us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:55.497064 (+ 121us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":234494720}
I20260809 15:52:55.420300 14300 raft_consensus.cc:729] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:55.497035 14312 consensus_queue.cc:260] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.481046 14237 raft_consensus.cc:2726] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:55.487603 10927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52758 (request call id 801) took 1808 ms. Trace:
I20260809 15:52:55.516225 10927 rpcz_store.cc:276] 0809 15:52:53.678701 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:53.678813 (+ 112us) service_pool.cc:227] Handling call
0809 15:52:53.783664 (+104851us) spinlock_profiling.cc:243] Waited 11.1 ms on lock 0x7b58000d1498. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f7966593851 00007f7966593702 00007f796f483b09 00007f796e3bf730 00007f796e48579b 00007f796e485821 00007f796e4351b0 00007f796e434a94 00007f7967b1cc04 00007f7967b1cb92
0809 15:52:55.487426 (+1703762us) spinlock_profiling.cc:243] Waited 517 ms on lock 0x7b540021acd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:55.487543 (+ 117us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1215560320}
I20260809 15:52:55.472625 11018 raft_consensus.cc:1261] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260809 15:52:55.517158 11018 pending_rounds.cc:76] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa: Aborting all ops after (but not including) 4
I20260809 15:52:55.517401 11018 pending_rounds.cc:97] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260809 15:52:55.517678 11018 raft_consensus.cc:2866] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:55.488566 13979 raft_consensus.cc:3037] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:55.441649 14223 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:55.521303 14163 consensus_queue.cc:260] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:55.522619 14163 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
W20260809 15:52:55.518924 14229 consensus_meta.cc:287] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.642s user 0.011s sys 0.008s
I20260809 15:52:55.523229 14229 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.525568 14229 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:55.412422 14194 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.493613 14305 consensus_queue.cc:1225] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer a53441872c0c416b8e2f26f52399f536 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
I20260809 15:52:55.529254 11010 raft_consensus.cc:1261] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260809 15:52:55.511220 10839 raft_consensus.cc:2454] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:55.563128 10926 raft_consensus.cc:1226] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:55.567991 11017 raft_consensus.cc:1226] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8. Ops: []
I20260809 15:52:55.559129 14094 raft_consensus.cc:2781] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:55.552812 14278 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:55.593420 14320 log.cc:924] Time spent T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.154s user 0.000s sys 0.000s
I20260809 15:52:55.589156 10893 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
W20260809 15:52:55.615088 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 327035us
I20260809 15:52:55.615887 10978 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:55.615897 11016 raft_consensus.cc:1226] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:55.564817 14237 consensus_queue.cc:1035] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260809 15:52:55.567716 14049 consensus_queue.cc:1035] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.048s
W20260809 15:52:55.626878 14316 log.cc:924] Time spent T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.111s user 0.001s sys 0.000s
I20260809 15:52:55.648655 10978 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:55.577307 14188 consensus_queue.cc:1046] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 4.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.041s
I20260809 15:52:55.626462 14228 raft_consensus.cc:2781] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election won for term 4
W20260809 15:52:55.659924 14304 log.cc:924] Time spent T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.774s user 0.007s sys 0.000s
I20260809 15:52:55.655210 10922 raft_consensus.cc:1261] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 15:52:55.600795 14278 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.722602 13992 raft_consensus.cc:3032] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:55.723245 13992 raft_consensus.cc:729] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:55.724404 13992 consensus_queue.cc:260] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:55.727902 14312 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:55.680635 14095 raft_consensus.cc:2726] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:55.641661 13656 consensus_queue.cc:1046] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.048s
I20260809 15:52:55.743268 10978 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:55.744532 10978 leader_election.cc:302] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:55.747407 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:55.748049 10922 raft_consensus.cc:2362] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6. Re-sending same reply.
I20260809 15:52:55.749148 14306 raft_consensus.cc:2726] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:55.688798 14325 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:55.751804 14229 raft_consensus.cc:2781] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:52:55.752556 14229 raft_consensus.cc:686] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:55.615520 14300 consensus_queue.cc:260] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:55.759735 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:55.762533 10922 raft_consensus.cc:2379] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 3: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:55.700680 13917 raft_consensus.cc:2726] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:55.764669 10922 raft_consensus.cc:1261] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 15:52:55.765388 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:55.693987 14228 raft_consensus.cc:686] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:55.766163 10921 raft_consensus.cc:2379] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 3: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:55.767910 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:55.767584 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 11 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:55.781778 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:55.783838 10978 leader_election.cc:302] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:55.775838 14318 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:55.777562 14095 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:52:55.795258 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 149970us
I20260809 15:52:55.800189 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:55.802879 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:55.806784 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 169699us
I20260809 15:52:55.753798 14229 consensus_queue.cc:237] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:55.783802 13917 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:55.812216 11009 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:55.787578 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:55.827083 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:55.801693 10843 raft_consensus.cc:2932] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 4 to 6, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:55.828305 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:55.829685 10842 raft_consensus.cc:1261] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 15:52:55.791107 14297 raft_consensus.cc:3037] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:55.766009 14242 consensus_queue.cc:1046] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.026s
I20260809 15:52:55.840348 10844 raft_consensus.cc:2379] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
W20260809 15:52:55.841312 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 1.36545e+06us
I20260809 15:52:55.844457 11010 raft_consensus.cc:1226] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:55.851743 11018 raft_consensus.cc:1261] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 15:52:55.766062 14293 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:55.859594 14260 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:55.860180 14260 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.862533 13971 raft_consensus.cc:2781] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:55.863154 14278 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:55.867516 11012 raft_consensus.cc:1226] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 4. Current term is 5. Ops: []
W20260809 15:52:55.868225 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 1.34936e+06us
I20260809 15:52:55.815582 13806 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:52:55.881512 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 74946.1us
I20260809 15:52:55.837505 14280 raft_consensus.cc:3032] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:55.872267 11009 raft_consensus.cc:1261] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260809 15:52:55.839884 14167 raft_consensus.cc:2781] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:52:55.893589 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:55.986850 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 55073.6us
I20260809 15:52:55.893251 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:55.987498 11641 raft_consensus.cc:2454] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 2.
I20260809 15:52:55.892434 11606 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:55.932175 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:55.988314 10926 raft_consensus.cc:1261] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260809 15:52:55.988469 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:55.932279 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:55.987124 10978 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:55.990134 11642 raft_consensus.cc:2396] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260809 15:52:55.989319 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:55.852380 13299 raft_consensus.cc:3032] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:55.992090 11639 raft_consensus.cc:1261] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260809 15:52:55.852577 14256 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:55.864907 14188 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:56.013092 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:56.014046 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:55.877418 14318 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:55.863204 13971 raft_consensus.cc:686] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:56.006815 13299 raft_consensus.cc:729] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:56.024734 14047 raft_consensus.cc:2781] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:56.024590 14169 consensus_queue.cc:1046] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:55.993428 14247 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.025776 11604 leader_election.cc:398] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Vote denied by peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223) with higher term. Message: Invalid argument: T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3.
I20260809 15:52:56.056028 11604 leader_election.cc:401] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 2 election: Cancelling election due to peer responding with higher term
I20260809 15:52:56.025359 14188 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:56.026401 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 134080us
I20260809 15:52:56.026785 11632 raft_consensus.cc:1226] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:56.027137 14004 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.021781 13913 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:56.027729 13958 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:56.028290 14192 raft_consensus.cc:2781] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:52:56.028856 14167 raft_consensus.cc:686] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:55.817662 14262 consensus_queue.cc:1035] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.169s
I20260809 15:52:56.045346 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
W20260809 15:52:56.023523 10811 leader_election.cc:334] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 2.068s (SENT)
I20260809 15:52:55.903301 14280 raft_consensus.cc:729] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:55.987421 13906 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:55.889624 14194 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:55.839138 13917 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.052719 14047 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:55.813001 14095 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:55.991847 14220 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:55.813594 14228 consensus_queue.cc:237] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.137218 14220 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.139425 14220 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:56.068197 14261 consensus_meta.cc:287] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.586s user 0.010s sys 0.002s
W20260809 15:52:56.140537 14267 consensus_meta.cc:287] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.642s user 0.006s sys 0.006s
I20260809 15:52:55.876257 14123 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:56.141557 13917 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:56.142397 14123 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:56.144243 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:56.146221 11637 raft_consensus.cc:2362] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6. Re-sending same reply.
I20260809 15:52:56.148392 14287 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Advancing to term 9
W20260809 15:52:56.069408 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 213817us
I20260809 15:52:56.044435 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:56.026997 14335 consensus_queue.cc:1035] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:56.045768 14260 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:55.890946 13806 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.150373 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
W20260809 15:52:56.150899 14276 log.cc:924] Time spent T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 1.410s user 0.006s sys 0.022s
I20260809 15:52:56.151846 13806 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:56.058969 13999 consensus_meta.cc:287] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.565s user 0.006s sys 0.004s
I20260809 15:52:56.013820 14033 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.064s
I20260809 15:52:56.087721 10811 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:56.090070 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:56.155638 10927 raft_consensus.cc:2396] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260809 15:52:56.092864 10837 raft_consensus.cc:1261] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260809 15:52:56.098459 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 72813.7us
I20260809 15:52:56.022652 14256 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:56.158131 14133 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.158672 14133 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.159578 14256 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:56.081938 13992 raft_consensus.cc:3037] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:56.160125 14094 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:56.160748 14094 raft_consensus.cc:3037] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:56.162551 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 2 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:56.086114 13958 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.101140 14164 consensus_queue.cc:1035] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 3, Time since last communication: 0.069s
I20260809 15:52:56.089157 13913 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.164662 13958 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:56.165162 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:56.166035 11012 raft_consensus.cc:2396] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 6 because replica has last-logged OpId of term: 4 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 6.
I20260809 15:52:56.166199 13913 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:56.055968 14279 consensus_queue.cc:1046] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.053s
I20260809 15:52:56.165525 11640 raft_consensus.cc:2932] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Committing config change with OpId 5.6: config changed from index 4 to 6, c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:52:56.069535 14247 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:56.082633 14004 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.156807 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:56.085737 14192 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:56.083993 14005 consensus_queue.cc:1046] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.039s
I20260809 15:52:56.056589 14318 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:56.106594 14334 log.cc:924] Time spent T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.116s user 0.000s sys 0.000s
I20260809 15:52:56.102807 14174 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:56.107877 14171 raft_consensus.cc:482] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.108314 14259 raft_consensus.cc:3037] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:56.105161 13906 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.178175 14192 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:56.122460 11009 raft_consensus.cc:1226] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:56.121407 10920 raft_consensus.cc:1226] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:56.111482 14325 raft_consensus.cc:2726] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:56.121593 14048 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.098335 14169 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.114329 13971 consensus_queue.cc:237] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 1.3, Last appended by leader: 3, Current term: 4, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.122924 14280 consensus_queue.cc:260] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:56.119657 14216 consensus_meta.cc:287] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.656s user 0.005s sys 0.004s
I20260809 15:52:56.133555 14095 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:56.135246 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:56.103811 14047 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:56.122967 13299 consensus_queue.cc:260] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.123719 10921 raft_consensus.cc:1226] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 4. Current term is 5. Ops: []
W20260809 15:52:56.064337 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 44734.1us
I20260809 15:52:56.140175 14261 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.274609 10924 raft_consensus.cc:2396] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for term 6 because replica has last-logged OpId of term: 5 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260809 15:52:55.890007 14031 consensus_queue.cc:1046] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.169s
I20260809 15:52:56.107813 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:56.152587 10929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 2026) took 1656 ms. Trace:
I20260809 15:52:56.079200 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:56.164707 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:56.169553 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:56.097553 14327 log.cc:924] Time spent T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.333s user 0.001s sys 0.000s
W20260809 15:52:56.107439 14324 log.cc:924] Time spent T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.086s user 0.000s sys 0.000s
W20260809 15:52:56.182742 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 159242us
W20260809 15:52:56.295775 10811 leader_election.cc:334] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 2.101s (SENT)
I20260809 15:52:56.182924 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:56.213637 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:56.107158 14293 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:56.179587 14223 consensus_meta.cc:287] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.648s user 0.011s sys 0.000s
W20260809 15:52:56.112331 13979 consensus_meta.cc:287] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.585s user 0.008s sys 0.000s
I20260809 15:52:56.117872 14188 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:56.246218 14329 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.305797 10844 raft_consensus.cc:1226] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:56.218241 11641 raft_consensus.cc:1226] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
W20260809 15:52:56.124581 14326 log.cc:924] Time spent T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.349s user 0.001s sys 0.000s
I20260809 15:52:56.280001 14278 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.234856 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:56.144490 14297 raft_consensus.cc:2726] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:56.240595 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:56.311679 10837 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:56.312055 10842 raft_consensus.cc:2379] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 6: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:56.213572 14033 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.215167 14164 raft_consensus.cc:2726] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:56.227717 14285 raft_consensus.cc:3037] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:56.267843 10893 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:56.272639 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:56.280017 10929 rpcz_store.cc:276] 0809 15:52:54.495914 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:54.496504 (+ 590us) service_pool.cc:227] Handling call
0809 15:52:54.496835 (+ 331us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:54.497075 (+ 240us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 15:52:54.497100 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:54.572003 (+ 74903us) spinlock_profiling.cc:243] Waited 4.52 ms on lock 0x7b2c0000f228. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7967ed6e11 00007f7967ee6442 00007f796952ec11 00007f79694f358a 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772
0809 15:52:54.572363 (+ 360us) log.cc:842] Serialized 26 byte log entry
0809 15:52:54.616090 (+ 43727us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:54.616126 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:54.616206 (+ 80us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:56.150269 (+1534063us) spinlock_profiling.cc:243] Waited 8.19 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:52:56.152386 (+ 2117us) raft_consensus.cc:1671] finished
0809 15:52:56.152406 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:56.152533 (+ 127us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":29245952,"thread_start_us":40126,"threads_started":1,"wal-append.queue_time_us":94445}
I20260809 15:52:56.289143 11642 raft_consensus.cc:2379] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 4: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:56.290232 10978 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:56.290616 11010 raft_consensus.cc:2396] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 7 because replica has last-logged OpId of term: 5 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260809 15:52:56.239634 14338 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.301725 11017 raft_consensus.cc:2379] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 8: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
W20260809 15:52:56.302438 10811 leader_election.cc:334] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 2.113s (SENT)
I20260809 15:52:56.323638 10811 leader_election.cc:302] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:56.323889 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 56183.9us
I20260809 15:52:56.324604 10893 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:56.244042 14049 consensus_queue.cc:1046] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.060s
I20260809 15:52:56.220168 14171 raft_consensus.cc:504] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:56.116755 14163 consensus_meta.cc:287] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.594s user 0.005s sys 0.004s
I20260809 15:52:56.281957 14260 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.108376 14167 consensus_queue.cc:237] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.348714 10978 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:56.315677 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:56.349768 10921 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:56.320216 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:56.325947 10893 leader_election.cc:302] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:56.279131 14030 consensus_queue.cc:1046] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.034s
I20260809 15:52:56.245798 14048 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.351912 10893 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:56.223624 14174 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.352458 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:56.351400 14338 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.246335 14169 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.356752 10893 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:56.357108 10843 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:56.359896 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:56.359513 10978 leader_election.cc:302] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:56.366334 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:52:56.359301 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 56905.5us
I20260809 15:52:56.365960 11642 raft_consensus.cc:2379] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
W20260809 15:52:56.368042 10811 leader_election.cc:334] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 2.113s (SENT)
I20260809 15:52:56.368537 10811 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:56.354403 14260 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.316880 14278 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:56.377416 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:56.385592 11642 raft_consensus.cc:2454] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:56.355773 13979 raft_consensus.cc:2726] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:56.313054 14329 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:56.399279 10921 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
I20260809 15:52:56.371479 14308 raft_consensus.cc:2726] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 15:52:56.364058 14279 raft_consensus.cc:2726] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:56.348451 14228 raft_consensus.cc:2726] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:56.348169 14095 consensus_queue.cc:1035] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260809 15:52:56.302497 14229 consensus_queue.cc:1035] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.047s
I20260809 15:52:56.322592 14033 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.359163 14188 raft_consensus.cc:2726] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 15:52:56.344663 14297 raft_consensus.cc:2726] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:56.360868 14345 raft_consensus.cc:2726] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:56.344898 13958 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.396931 11635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 703) took 2647 ms. Trace:
I20260809 15:52:56.402508 10843 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 8.
I20260809 15:52:56.371976 14258 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:56.405162 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:56.406903 11012 raft_consensus.cc:1226] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 11 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 10. Current term is 11. Ops: []
W20260809 15:52:56.407064 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 60796.5us
W20260809 15:52:56.413826 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 54379us
I20260809 15:52:56.383895 10893 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:56.386682 14293 raft_consensus.cc:2741] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260809 15:52:56.414615 10923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 2034) took 1110 ms. Trace:
I20260809 15:52:56.396723 11632 raft_consensus.cc:3037] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:56.481184 14300 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:56.400038 14164 raft_consensus.cc:2726] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:56.482355 14258 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.483533 14004 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:56.484807 14171 leader_election.cc:288] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:56.466269 13906 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:56.487564 14260 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:56.470883 13958 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.493321 13958 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:56.435621 14247 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:56.494834 14349 log.cc:924] Time spent T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.110s user 0.000s sys 0.001s
I20260809 15:52:56.497788 14048 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:56.500494 11606 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:56.434118 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:56.360222 14268 consensus_queue.cc:1046] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
W20260809 15:52:56.439909 14343 log.cc:924] Time spent T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.123s user 0.000s sys 0.001s
I20260809 15:52:56.447947 14318 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:56.452368 14252 raft_consensus.cc:3032] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 15:52:56.432104 14338 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:56.456902 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:56.364332 14223 consensus_queue.cc:1035] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.053s
I20260809 15:52:56.471032 10923 rpcz_store.cc:276] 0809 15:52:55.304526 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:55.319791 (+ 15265us) service_pool.cc:227] Handling call
0809 15:52:55.383061 (+ 63270us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:55.383432 (+ 371us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:55.383459 (+ 27us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:55.383878 (+ 419us) log.cc:842] Serialized 26 byte log entry
0809 15:52:55.701410 (+317532us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:55.836736 (+135326us) log.cc:842] Serialized 14 byte log entry
0809 15:52:55.853279 (+ 16543us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:55.853375 (+ 96us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:56.414360 (+560985us) raft_consensus.cc:1671] finished
0809 15:52:56.414389 (+ 29us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:56.414542 (+ 153us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":133747,"prepare.queue_time_us":385675,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":41,"raft.queue_time_us":41111,"raft.run_cpu_time_us":333,"raft.run_wall_time_us":22257,"spinlock_wait_cycles":1792,"thread_start_us":339585,"threads_started":2,"wal-append.queue_time_us":422527}
I20260809 15:52:56.477770 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:56.530644 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:56.531317 11639 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:56.503674 10978 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:56.418807 11634 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:56.533499 10978 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:56.534585 14293 raft_consensus.cc:2726] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260809 15:52:56.535571 14293 raft_consensus.cc:2726] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
W20260809 15:52:56.507431 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 123619us
I20260809 15:52:56.538009 10978 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:56.538707 10893 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:56.477707 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:56.539484 14318 raft_consensus.cc:2726] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:52:56.539902 11642 raft_consensus.cc:2379] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:56.540933 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:56.541657 11642 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:56.435818 14325 raft_consensus.cc:3037] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:56.456210 14169 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:56.459424 11635 rpcz_store.cc:276] 0809 15:52:53.737660 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:53.737768 (+ 108us) service_pool.cc:227] Handling call
0809 15:52:53.768884 (+ 31116us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 15:52:53.969551 (+200667us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:53.969605 (+ 54us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 15:52:54.046511 (+ 76906us) log.cc:842] Serialized 52 byte log entry
0809 15:52:54.162430 (+115919us) raft_consensus.cc:1631] Marking committed up to 8
0809 15:52:54.382466 (+220036us) log.cc:842] Serialized 14 byte log entry
0809 15:52:55.818843 (+1436377us) spinlock_profiling.cc:243] Waited 1.11 ms on lock 0x7b18000b5900. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f79665424ca 00007f796fc23f99 00007f7966406990 00007f79665c8864 00007f79665c8155 00007f79694eb3d0 00007f79694fd576 00007f796950e0d7 00007f796950e058
0809 15:52:56.290750 (+471907us) log.cc:842] Serialized 14 byte log entry
0809 15:52:56.360749 (+ 69999us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:56.360828 (+ 79us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:56.370857 (+ 10029us) raft_consensus.cc:1671] finished
0809 15:52:56.370878 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:56.385404 (+ 14526us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":131336,"prepare.queue_time_us":526437,"prepare.run_cpu_time_us":110,"prepare.run_wall_time_us":1220,"raft.queue_time_us":1596148,"raft.run_cpu_time_us":291,"raft.run_wall_time_us":33865,"spinlock_wait_cycles":2561408,"thread_start_us":977303,"threads_started":6,"wal-append.queue_time_us":349909,"wal-append.run_cpu_time_us":4209,"wal-append.run_wall_time_us":1474548}
I20260809 15:52:56.542932 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:56.543336 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:56.543751 10841 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:52:56.544265 11635 raft_consensus.cc:2379] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 7: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:56.545261 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:56.465471 14215 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.546650 14215 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.547180 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:56.549968 14215 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:56.545835 11635 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Advancing to term 11
W20260809 15:52:56.555913 10978 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.512s (SENT) [suppressed 80 similar messages]
I20260809 15:52:56.556612 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:56.558121 10843 raft_consensus.cc:1226] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260809 15:52:56.564059 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:56.520493 14048 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.518234 14261 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:56.528172 14252 raft_consensus.cc:729] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:56.527719 14171 raft_consensus.cc:2726] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:56.589871 11642 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
W20260809 15:52:56.568954 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 60247.8us
W20260809 15:52:56.586655 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 180766us
I20260809 15:52:56.586807 11636 raft_consensus.cc:1226] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:56.559975 13906 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.564085 14228 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.593611 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:56.596150 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:56.594907 11639 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:52:56.622177 11012 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:56.595342 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:56.581086 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 193513us
I20260809 15:52:56.622913 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:56.623533 11015 raft_consensus.cc:2396] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260809 15:52:56.526974 14004 consensus_queue.cc:1035] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.057s
W20260809 15:52:56.623456 10811 leader_election.cc:334] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 2.221s (SENT)
I20260809 15:52:56.576489 14174 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:56.518784 14297 consensus_queue.cc:1035] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.056s
I20260809 15:52:56.631232 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:56.585067 14267 raft_consensus.cc:2726] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 15:52:56.578629 14164 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.602854 14325 raft_consensus.cc:2726] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:56.647605 11012 raft_consensus.cc:2379] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 3: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:56.642138 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:56.634990 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:56.646042 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:56.620064 14259 raft_consensus.cc:2726] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election lost for term 2. Reason: Vote denied by peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223) with higher term. Message: Invalid argument: T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3.
I20260809 15:52:56.568325 14094 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.581089 14047 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:56.673053 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 71781.4us
I20260809 15:52:56.651674 11639 pending_rounds.cc:76] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9: Aborting all ops after (but not including) 5
W20260809 15:52:56.673604 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 87662.8us
W20260809 15:52:56.727756 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 227398us
I20260809 15:52:56.761579 11639 pending_rounds.cc:97] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260809 15:52:56.729024 11011 raft_consensus.cc:1226] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8. Ops: []
I20260809 15:52:56.626258 13971 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:52:56.736428 14366 log.cc:924] Time spent T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.098s user 0.000s sys 0.000s
I20260809 15:52:56.583962 14048 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.762565 13971 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.765784 14048 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:56.766129 13971 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:56.767400 11606 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:56.770807 11013 raft_consensus.cc:1226] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:56.742034 14033 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:56.760411 11635 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 11.
I20260809 15:52:56.762005 11639 raft_consensus.cc:2866] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:56.623971 13906 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.630712 14228 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.795539 10893 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:56.650597 14164 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.779084 11635 raft_consensus.cc:1226] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 8 LEADER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8. Ops: []
I20260809 15:52:56.804122 10893 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:56.828137 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 67683.3us
I20260809 15:52:56.791226 14308 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.799896 14270 raft_consensus.cc:2726] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260809 15:52:56.837251 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 164955us
I20260809 15:52:56.838836 11635 raft_consensus.cc:1226] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:56.841697 10811 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:56.807685 14249 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.815807 13999 raft_consensus.cc:2781] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:56.832733 14285 raft_consensus.cc:2726] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260809 15:52:56.861277 14367 log.cc:924] Time spent T 0b9fb9f88a6847af87fef710d1a2793b P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.058s user 0.000s sys 0.000s
I20260809 15:52:56.837284 14329 leader_election.cc:288] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:56.866881 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:56.872069 10811 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:56.846679 14172 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.801005 14342 consensus_queue.cc:1046] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true }, Status: INVALID_TERM, Last received: 10.10, Next index: 11, Last known committed idx: 9, Time since last communication: 0.063s
W20260809 15:52:56.877661 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 321761us
W20260809 15:52:56.880779 13992 consensus_meta.cc:287] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.721s user 0.007s sys 0.004s
I20260809 15:52:56.854689 13999 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:56.897637 13999 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:56.856333 14247 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.902355 14261 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260809 15:52:56.903038 14261 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:56.902592 14247 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.903664 14261 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Advancing to term 11
W20260809 15:52:56.888468 10978 leader_election.cc:334] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.553s (SENT)
I20260809 15:52:56.906809 10978 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:56.896310 10811 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
W20260809 15:52:56.874130 14192 consensus_meta.cc:287] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.629s user 0.015s sys 0.000s
I20260809 15:52:56.914193 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:56.915057 11013 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
W20260809 15:52:56.919248 10978 leader_election.cc:334] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.575s (SENT)
W20260809 15:52:56.921272 10978 leader_election.cc:334] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.779s (SENT)
I20260809 15:52:56.921891 10978 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:56.922220 10921 raft_consensus.cc:1261] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 6 index: 5. (index mismatch)
I20260809 15:52:56.926117 11642 raft_consensus.cc:1226] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4. Ops: []
I20260809 15:52:56.926956 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:56.927260 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:56.874739 14047 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:56.940716 10927 raft_consensus.cc:2379] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:56.844113 14308 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:57.024186 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 114787us
I20260809 15:52:57.024350 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:57.024626 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
W20260809 15:52:56.913478 14360 log.cc:924] Time spent T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.293s user 0.000s sys 0.001s
W20260809 15:52:57.073912 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 177667us
I20260809 15:52:57.061688 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:57.083565 10931 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:56.897006 14033 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:56.916435 14049 raft_consensus.cc:2726] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:56.913810 14285 raft_consensus.cc:482] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:57.009019 14279 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:57.011463 14297 raft_consensus.cc:2726] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:56.929828 14290 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.929847 14318 raft_consensus.cc:2726] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:57.191756 10843 raft_consensus.cc:1226] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: []
I20260809 15:52:57.023937 14306 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:56.855044 14249 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.071947 13806 raft_consensus.cc:2726] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:56.905896 14172 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:56.994328 10978 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:57.224395 14249 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:57.113190 14325 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:57.130740 14094 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:56.926095 14192 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.238674 11636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 938) took 1151 ms. Trace:
I20260809 15:52:57.131810 13906 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:57.239691 14325 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.249675 11636 rpcz_store.cc:276] 0809 15:52:56.079069 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:56.079302 (+ 233us) service_pool.cc:227] Handling call
0809 15:52:56.079615 (+ 313us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:56.593112 (+513497us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:57.229829 (+636717us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b54007dafd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:57.230110 (+ 281us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1165911808}
I20260809 15:52:57.239146 14249 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:57.249528 11018 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.791s user 0.002s sys 0.009s
W20260809 15:52:57.261343 11018 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.803s user 0.002s sys 0.009s
I20260809 15:52:57.193383 14285 raft_consensus.cc:504] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:57.248808 14369 log.cc:924] Time spent T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.503s user 0.000s sys 0.001s
I20260809 15:52:57.132298 14033 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:57.227840 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 297440us
I20260809 15:52:57.232661 14297 raft_consensus.cc:2781] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election won for term 8
W20260809 15:52:57.270329 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 196231us
I20260809 15:52:57.231238 11633 raft_consensus.cc:2454] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:57.299389 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:57.198365 14279 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.313583 11014 raft_consensus.cc:1226] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:57.226035 14290 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:57.296612 10931 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 8.
I20260809 15:52:57.314617 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:57.318704 11013 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:57.315719 14047 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:57.296864 11634 consensus_meta.cc:287] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.764s user 0.013s sys 0.000s
I20260809 15:52:57.322118 10841 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:57.229903 14306 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.332927 14047 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:57.336731 14047 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:57.337842 11634 raft_consensus.cc:2454] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
W20260809 15:52:57.342373 11605 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.758s user 0.007s sys 0.004s
W20260809 15:52:57.342782 11605 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.830s user 0.007s sys 0.004s
I20260809 15:52:57.345530 14279 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:57.347172 14380 log.cc:924] Time spent T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.092s user 0.000s sys 0.000s
I20260809 15:52:57.347679 10841 raft_consensus.cc:2454] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
W20260809 15:52:57.348624 14387 log.cc:924] Time spent T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.089s user 0.000s sys 0.000s
I20260809 15:52:57.359483 14172 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:57.362975 14033 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:57.365175 14285 leader_election.cc:288] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:57.291905 14164 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:57.372872 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 232377us
W20260809 15:52:57.378114 14369 log.cc:924] Time spent T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.065s user 0.000s sys 0.001s
W20260809 15:52:57.376255 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 67516.2us
I20260809 15:52:57.298564 14228 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:57.313655 14335 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:57.391211 14335 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.295249 14318 consensus_queue.cc:1046] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
I20260809 15:52:57.321388 14383 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:57.392792 14383 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.378789 14048 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:57.295794 14247 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:57.293447 14249 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.390563 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:57.379786 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 304961us
I20260809 15:52:57.394732 14192 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:57.389850 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:57.291582 13806 consensus_queue.cc:1035] T 6508f17228904310be3c95818973f3a1 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.082s
I20260809 15:52:57.394958 14383 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:57.323632 14284 consensus_queue.cc:1046] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.060s
I20260809 15:52:57.417246 11011 raft_consensus.cc:2454] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:57.389979 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:57.424642 11637 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41488 (request call id 925) took 1532 ms (client timeout 1791 ms). Trace:
I20260809 15:52:57.422346 14335 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:57.383802 14172 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
W20260809 15:52:57.425103 11637 rpcz_store.cc:269] 0809 15:52:55.892515 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:55.892645 (+ 130us) service_pool.cc:227] Handling call
0809 15:52:57.424461 (+1531816us) spinlock_profiling.cc:243] Waited 124 ms on lock 0x7b5400089654. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:57.424583 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":285552896}
I20260809 15:52:57.426548 10891 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:57.427181 11639 raft_consensus.cc:1261] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260809 15:52:57.427790 11632 consensus_meta.cc:287] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.949s user 0.009s sys 0.005s
I20260809 15:52:57.456269 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:57.428619 14306 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:57.424558 11015 raft_consensus.cc:2379] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 6: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:57.459893 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:57.457036 11009 raft_consensus.cc:2454] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:57.460842 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:57.461714 11017 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:57.455997 10809 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:57.463054 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:57.465207 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:57.465935 10930 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 8.
I20260809 15:52:57.467068 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:57.467731 10930 raft_consensus.cc:2454] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
W20260809 15:52:57.429730 14287 consensus_meta.cc:287] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 1.276s user 0.008s sys 0.007s
I20260809 15:52:57.474629 14033 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:52:57.474624 10811 connection.cc:232] RPC call timeout handler was delayed by -1.037142988992855s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1986s for a response
I20260809 15:52:57.470923 10927 raft_consensus.cc:2396] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260809 15:52:57.429088 14249 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:57.472856 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:57.476958 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:57.477860 11637 raft_consensus.cc:1226] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 5. Current term is 6. Ops: [5.7-5.7]
I20260809 15:52:57.554713 14216 raft_consensus.cc:3032] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:57.574791 14220 raft_consensus.cc:3032] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:52:57.536731 14396 raft_consensus.cc:2781] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:57.470783 14335 consensus_queue.cc:1046] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.038s
I20260809 15:52:57.574189 13299 raft_consensus.cc:3037] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:57.547832 14252 consensus_queue.cc:260] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.464329 14172 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:57.552620 14359 log.cc:924] Time spent T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.914s user 0.006s sys 0.004s
I20260809 15:52:57.635645 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:57.637117 11010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2409) took 1361 ms. Trace:
I20260809 15:52:57.637640 11010 rpcz_store.cc:276] 0809 15:52:56.275430 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:56.355015 (+ 79585us) service_pool.cc:227] Handling call
0809 15:52:56.355354 (+ 339us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:56.459218 (+103864us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:56.459258 (+ 40us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:56.459285 (+ 27us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:52:56.459332 (+ 47us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:57.636818 (+1177486us) spinlock_profiling.cc:243] Waited 104 ms on lock 0x7b540040b354. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:57.636879 (+ 61us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:57.637042 (+ 163us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":238309248}
I20260809 15:52:57.638959 11641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 941) took 1513 ms. Trace:
I20260809 15:52:57.639422 11641 rpcz_store.cc:276] 0809 15:52:56.125512 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:56.130342 (+ 4830us) service_pool.cc:227] Handling call
0809 15:52:56.130650 (+ 308us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:56.306460 (+175810us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:57.638658 (+1332198us) spinlock_profiling.cc:243] Waited 87.3 ms on lock 0x7b5400719dd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:57.638890 (+ 232us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":200826240}
I20260809 15:52:57.626657 14216 raft_consensus.cc:729] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:57.626302 14306 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:52:57.548750 13999 consensus_meta.cc:287] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.650s user 0.011s sys 0.000s
I20260809 15:52:57.556286 11642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 943) took 1430 ms. Trace:
I20260809 15:52:57.584260 10921 raft_consensus.cc:1261] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 3. (term mismatch)
I20260809 15:52:57.650962 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:57.650655 13999 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.651855 10929 raft_consensus.cc:2454] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:57.652753 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:57.653569 13999 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:57.650745 10921 pending_rounds.cc:76] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3: Aborting all ops after (but not including) 2
I20260809 15:52:57.654373 10921 pending_rounds.cc:97] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3: Aborting uncommitted NO_OP operation due to leader change: 5.3
I20260809 15:52:57.653767 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:57.654698 10921 raft_consensus.cc:2866] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:57.627187 14220 raft_consensus.cc:729] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:57.650640 11642 rpcz_store.cc:276] 0809 15:52:56.126267 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:56.632784 (+506517us) service_pool.cc:227] Handling call
0809 15:52:56.633120 (+ 336us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:56.930032 (+296912us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:57.555972 (+625940us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b5400694bd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:57.556207 (+ 235us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":673280768}
I20260809 15:52:57.656666 11642 raft_consensus.cc:1261] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 7. (index mismatch)
W20260809 15:52:57.545568 14261 consensus_meta.cc:287] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.641s user 0.014s sys 0.000s
I20260809 15:52:57.561385 14405 consensus_queue.cc:1035] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
W20260809 15:52:57.583433 10924 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 2069) took 2576 ms (client timeout 1589 ms). Trace:
I20260809 15:52:57.557967 11639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 949) took 1429 ms. Trace:
W20260809 15:52:57.551172 14300 consensus_meta.cc:287] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 1.042s user 0.003s sys 0.009s
I20260809 15:52:57.582321 14319 raft_consensus.cc:3032] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 10 LEADER]: Stepping down as leader of term 10
I20260809 15:52:57.662359 14319 raft_consensus.cc:729] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 10 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:57.663317 14319 consensus_queue.cc:260] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 10.10, Last appended by leader: 10, Current term: 10, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.664618 14319 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Advancing to term 11
I20260809 15:52:57.666576 10843 raft_consensus.cc:1261] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 15:52:57.513250 14389 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:57.667603 14389 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:57.670279 14262 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:57.670941 14262 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:57.559126 10844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 797) took 1481 ms. Trace:
I20260809 15:52:57.672295 10844 rpcz_store.cc:276] 0809 15:52:56.077886 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:56.078536 (+ 650us) service_pool.cc:227] Handling call
0809 15:52:56.088240 (+ 9704us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:56.306201 (+217961us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:57.558755 (+1252554us) spinlock_profiling.cc:243] Waited 215 ms on lock 0x7b540037ebd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:57.559043 (+ 288us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":494845824}
I20260809 15:52:57.519246 14325 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:57.469461 11632 raft_consensus.cc:2454] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:52:57.498198 10811 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:57.415508 14048 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:57.674605 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 176549us
I20260809 15:52:57.676061 14048 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:57.676640 10811 leader_election.cc:334] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.655s (SENT)
I20260809 15:52:57.676827 14048 raft_consensus.cc:2781] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election won for term 7
W20260809 15:52:57.546754 10877 heartbeater.cc:644] Failed to heartbeat to 127.9.84.62:42773 (0 consecutive failures): Timed out: Failed to send heartbeat to master: TSHeartbeat RPC to 127.9.84.62:42773 timed out after 15.000s (SENT)
I20260809 15:52:57.500344 14297 raft_consensus.cc:686] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:57.678328 14297 consensus_queue.cc:237] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:57.679152 14278 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:57.521174 14401 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:57.470232 14308 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:57.558493 14392 log.cc:924] Time spent T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.050s user 0.001s sys 0.000s
I20260809 15:52:57.680083 14401 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.547206 14258 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:57.492588 11017 raft_consensus.cc:2379] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 8: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
W20260809 15:52:57.677450 10811 leader_election.cc:334] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.659s (SENT)
I20260809 15:52:57.475427 14033 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:57.681790 10811 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:57.682287 14401 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:57.682608 14048 raft_consensus.cc:686] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 7 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:57.683179 11018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2169) took 1501 ms. Trace:
I20260809 15:52:57.683336 14033 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:57.497833 14267 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:57.684101 14267 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.683602 11018 rpcz_store.cc:276] 0809 15:52:56.181653 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:56.317354 (+135701us) service_pool.cc:227] Handling call
0809 15:52:56.359850 (+ 42496us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:56.405668 (+ 45818us) raft_consensus.cc:1520] Early marking committed up to index 3
0809 15:52:56.405697 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:56.406151 (+ 454us) log.cc:842] Serialized 26 byte log entry
0809 15:52:57.261796 (+855645us) raft_consensus.cc:1631] Marking committed up to 3
0809 15:52:57.261876 (+ 80us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:57.261963 (+ 87us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:57.682977 (+421014us) raft_consensus.cc:1671] finished
0809 15:52:57.682997 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:57.683118 (+ 121us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":66490,"raft.queue_time_us":54218,"raft.run_cpu_time_us":150,"raft.run_wall_time_us":14252,"thread_start_us":803585,"threads_started":1,"wal-append.queue_time_us":988734}
I20260809 15:52:57.477861 14094 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:57.455300 14383 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:57.535627 14290 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:57.601599 10927 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
W20260809 15:52:57.610477 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 843132us
I20260809 15:52:57.610684 13913 raft_consensus.cc:2726] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:57.694237 14389 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:57.615633 14167 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:57.424505 14228 consensus_queue.cc:1046] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.066s
I20260809 15:52:57.697695 14133 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:57.493996 10891 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:57.624935 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 151915us
W20260809 15:52:57.626168 10893 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.503s user 0.022s sys 0.040s
W20260809 15:52:57.702338 10893 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.630s user 0.023s sys 0.041s
W20260809 15:52:57.702725 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 630407us
I20260809 15:52:57.630277 11637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41478 (request call id 1018) took 1302 ms. Trace:
I20260809 15:52:57.706255 11637 rpcz_store.cc:276] 0809 15:52:56.326373 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:57.455090 (+1128717us) service_pool.cc:227] Handling call
0809 15:52:57.472477 (+ 17387us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:57.627362 (+154885us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:57.627604 (+ 242us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:52:57.533255 14318 raft_consensus.cc:3032] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:52:57.628084 14396 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:57.644515 11641 raft_consensus.cc:1226] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 11 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 10. Current term is 11. Ops: [8.9-10.10]
I20260809 15:52:57.641263 14213 raft_consensus.cc:3037] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:57.642709 14296 raft_consensus.cc:3032] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:52:57.647323 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:57.628994 14402 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:52:57.660053 10924 rpcz_store.cc:269] 0809 15:52:55.006991 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:55.008199 (+ 1208us) service_pool.cc:227] Handling call
0809 15:52:57.583193 (+2574994us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b540025b7d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:57.583342 (+ 149us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2904235776}
I20260809 15:52:57.661470 14261 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.471925 14391 raft_consensus.cc:2781] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:57.724474 14167 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.791419 14391 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:57.791505 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:57.791886 14391 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:57.792151 10847 raft_consensus.cc:2454] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:57.793088 14167 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:57.685801 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:57.793776 11010 raft_consensus.cc:2454] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:57.660148 11639 rpcz_store.cc:276] 0809 15:52:56.128056 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:57.130772 (+1002716us) service_pool.cc:227] Handling call
0809 15:52:57.426815 (+296043us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:57.455077 (+ 28262us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:57.557740 (+102663us) spinlock_profiling.cc:243] Waited 293 ms on lock 0x7b5400089650. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:57.557901 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":674505472}
I20260809 15:52:57.686282 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:57.795655 11017 raft_consensus.cc:2379] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 6: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:57.655984 14252 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:57.688459 14094 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:57.691303 14220 consensus_queue.cc:260] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.829913 10927 raft_consensus.cc:1261] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260809 15:52:57.656160 14306 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:57.832484 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 145132us
I20260809 15:52:57.676395 14172 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:57.693132 14401 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:57.688323 14383 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:57.694260 14254 consensus_queue.cc:1046] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 15:52:57.711701 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:57.714547 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 220642us
W20260809 15:52:57.718569 10893 leader_election.cc:334] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.791s (SENT)
I20260809 15:52:57.725696 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:57.685078 14048 consensus_queue.cc:237] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:57.782570 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:57.836755 10839 raft_consensus.cc:2454] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:57.733963 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:57.788513 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:57.729075 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:57.728497 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:57.838109 11637 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:57.785435 14296 raft_consensus.cc:729] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:57.797600 10847 raft_consensus.cc:1261] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 6 index: 3. (index mismatch)
I20260809 15:52:57.792017 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:57.838774 10844 raft_consensus.cc:2454] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:57.801983 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:57.802381 10809 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:57.804088 14419 log.cc:924] Time spent T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.080s user 0.001s sys 0.000s
I20260809 15:52:57.729423 14194 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:57.725558 14223 consensus_queue.cc:1046] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.079s
I20260809 15:52:57.815243 14030 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:52:57.819365 11634 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41488 (request call id 948) took 1691 ms (client timeout 1590 ms). Trace:
I20260809 15:52:57.820247 13958 raft_consensus.cc:3032] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:52:57.851473 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:57.851656 13958 raft_consensus.cc:729] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:57.851996 11013 raft_consensus.cc:1226] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:57.852664 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:57.853122 14094 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:57.852630 13958 consensus_queue.cc:260] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.853639 10839 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:57.822619 11642 raft_consensus.cc:1226] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260809 15:52:57.854246 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:55.295253 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:52:57.825713 10838 raft_consensus.cc:1226] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 11 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 10. Current term is 11. Ops: []
I20260809 15:52:57.856956 10847 raft_consensus.cc:2454] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:57.854586 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:57.854914 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:57.858937 11606 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:57.711524 14318 raft_consensus.cc:729] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:57.688745 14300 raft_consensus.cc:2726] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 15:52:57.727690 14396 raft_consensus.cc:3037] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:57.689937 14405 raft_consensus.cc:482] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:52:57.833209 10811 leader_election.cc:334] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.709s (SENT)
I20260809 15:52:57.725659 14188 raft_consensus.cc:2781] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:57.727092 14325 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:57.835539 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:57.795702 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:57.839370 10848 raft_consensus.cc:2379] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 5: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:57.795980 14402 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.838251 11641 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
I20260809 15:52:57.835741 10842 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 8.
W20260809 15:52:57.844182 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 125637us
I20260809 15:52:57.844448 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:57.848678 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:57.850404 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:57.836382 14401 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.822026 11640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 942) took 1696 ms. Trace:
I20260809 15:52:57.826676 14261 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 11 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:57.858693 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
W20260809 15:52:57.826892 11632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41488 (request call id 940) took 1701 ms (client timeout 1927 ms). Trace:
I20260809 15:52:57.727522 14368 consensus_queue.cc:1035] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.123s
I20260809 15:52:57.855865 10931 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:57.992727 11013 raft_consensus.cc:1226] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:57.981108 11637 raft_consensus.cc:2379] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:57.940150 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:57.909624 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:57.994781 11015 raft_consensus.cc:1261] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:52:57.994946 11009 raft_consensus.cc:2379] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 7: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:57.981817 11640 rpcz_store.cc:276] 0809 15:52:56.125851 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:56.130849 (+ 4998us) service_pool.cc:227] Handling call
0809 15:52:56.131125 (+ 276us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:56.165413 (+ 34288us) raft_consensus.cc:1520] Early marking committed up to index 7
0809 15:52:56.387078 (+221665us) log.cc:842] Serialized 14 byte log entry
0809 15:52:56.600930 (+213852us) log.cc:842] Serialized 14 byte log entry
0809 15:52:56.625600 (+ 24670us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:56.625635 (+ 35us) raft_consensus.cc:1631] Marking committed up to 7
0809 15:52:56.625679 (+ 44us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:57.609231 (+983552us) spinlock_profiling.cc:243] Waited 4.25 ms on lock 0x7b18002797c0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f79665424ca 00007f796fc23f99 00007f796d39bbcf 00007f79665cb7b6 00007f79665ceb11 00007f79665ceac9 00007f79665cea59 00007f79665cea21 00007f79665cdd1d
0809 15:52:57.821808 (+212577us) spinlock_profiling.cc:243] Waited 34.1 ms on lock 0x7b540083b154. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:57.821875 (+ 67us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:57.821977 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":92129,"prepare.queue_time_us":85272,"prepare.run_cpu_time_us":87,"prepare.run_wall_time_us":4924,"raft.queue_time_us":73580,"raft.run_cpu_time_us":336,"raft.run_wall_time_us":19359,"spinlock_wait_cycles":88109696,"thread_start_us":135040,"threads_started":1,"wal-append.queue_time_us":251593}
I20260809 15:52:57.903609 10926 raft_consensus.cc:1226] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:57.902733 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:57.856258 11010 raft_consensus.cc:1226] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8. Ops: []
I20260809 15:52:57.914351 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:57.855098 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:57.980784 11017 raft_consensus.cc:2454] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:57.967985 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:57.981977 11639 raft_consensus.cc:2454] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:58.001447 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:58.002328 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:57.981480 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 148300us
I20260809 15:52:57.982335 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:57.995889 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:58.013940 11639 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:52:58.002514 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:57.854053 13958 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:57.855646 14194 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.854308 13979 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:57.855446 14254 raft_consensus.cc:3032] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Stepping down as leader of term 2
I20260809 15:52:57.845510 14049 raft_consensus.cc:2781] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:58.004493 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:58.006778 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:58.025068 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:58.009539 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:58.010310 13925 raft_consensus.cc:3032] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:58.012715 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:58.025429 10926 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:58.014621 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:58.015132 10811 leader_election.cc:334] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 2.162s (SENT)
I20260809 15:52:57.981017 14188 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:58.016868 10891 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:58.016814 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:58.014294 11640 raft_consensus.cc:2362] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6. Re-sending same reply.
I20260809 15:52:58.017910 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:57.989895 10923 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 8.
I20260809 15:52:57.984413 14409 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:58.019423 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:57.998379 13913 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:58.023439 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.025341 10842 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 8.
I20260809 15:52:58.026244 10848 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
W20260809 15:52:58.009930 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 151051us
I20260809 15:52:58.027045 10811 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:58.027387 10929 raft_consensus.cc:2379] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 3: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:58.053896 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:58.054514 10929 raft_consensus.cc:2454] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:52:58.054872 13913 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.055672 10929 raft_consensus.cc:1261] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260809 15:52:58.027612 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:58.056236 14374 raft_consensus.cc:3032] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:57.903367 14030 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.057679 10847 raft_consensus.cc:2379] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 9: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:58.019025 14368 raft_consensus.cc:2781] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:57.993512 14405 raft_consensus.cc:504] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.918720 14283 consensus_queue.cc:1046] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.104s
I20260809 15:52:58.027817 13925 raft_consensus.cc:729] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:58.033196 14342 raft_consensus.cc:2781] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:58.016340 14400 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:58.053246 11641 raft_consensus.cc:2454] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:58.029884 10844 raft_consensus.cc:2379] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 5: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:57.854553 14296 consensus_queue.cc:260] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:57.850628 14391 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.056792 10926 raft_consensus.cc:2379] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 7: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
W20260809 15:52:57.851279 11634 rpcz_store.cc:269] 0809 15:52:56.127763 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:56.150061 (+ 22298us) service_pool.cc:227] Handling call
0809 15:52:57.819146 (+1669085us) spinlock_profiling.cc:243] Waited 262 ms on lock 0x7b54006f45d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:57.819279 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":602555136}
I20260809 15:52:58.057044 11606 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:58.039529 11639 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 9.
I20260809 15:52:58.096309 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.034539 10922 raft_consensus.cc:2396] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for term 5 because replica has last-logged OpId of term: 3 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260809 15:52:58.025070 14049 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:58.099947 14049 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:58.100447 14280 raft_consensus.cc:3037] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:57.991923 14325 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:58.059861 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 44747.5us
I20260809 15:52:58.033371 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.060685 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.061791 10929 raft_consensus.cc:1226] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260809 15:52:58.105536 11640 raft_consensus.cc:2362] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6. Re-sending same reply.
I20260809 15:52:58.056591 14374 raft_consensus.cc:729] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:58.059724 14368 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:58.016145 14252 consensus_queue.cc:1046] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.041s
I20260809 15:52:58.022876 14254 raft_consensus.cc:729] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 2 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:58.045677 10837 raft_consensus.cc:2379] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:58.025893 14261 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:58.120071 10811 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:58.073935 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:58.073161 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:58.080724 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:58.124969 11013 raft_consensus.cc:2379] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 11: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:58.073194 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:58.085904 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.081234 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:58.024585 14238 consensus_queue.cc:1035] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.039s
I20260809 15:52:58.089251 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:58.141691 10811 leader_election.cc:302] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:58.030452 14188 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:58.024688 13979 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:58.097775 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 44206.7us
I20260809 15:52:58.097148 14402 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.106344 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.132121 10920 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:54216 (request call id 1591) took 1222 ms. Trace:
I20260809 15:52:58.085225 14247 raft_consensus.cc:2726] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:58.113952 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:58.084990 14342 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:58.118930 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:58.184490 11017 raft_consensus.cc:2454] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:58.120826 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.185089 10847 raft_consensus.cc:1261] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260809 15:52:58.149502 11641 raft_consensus.cc:2454] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:52:58.117988 14296 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:58.150801 10842 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
W20260809 15:52:57.984872 11632 rpcz_store.cc:269] 0809 15:52:56.125130 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:56.134966 (+ 9836us) service_pool.cc:227] Handling call
0809 15:52:57.826722 (+1691756us) spinlock_profiling.cc:243] Waited 109 ms on lock 0x7b540066c454. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:57.826834 (+ 112us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":251121024}
I20260809 15:52:58.107277 14348 raft_consensus.cc:2781] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:58.171190 14133 raft_consensus.cc:2781] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 15:52:58.098862 14312 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:58.173724 11606 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:58.081996 13925 consensus_queue.cc:260] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.115160 14368 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:58.180284 11640 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 8.
I20260809 15:52:58.211459 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.103163 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:58.031843 14318 consensus_queue.cc:260] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:58.132213 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:58.186843 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.180750 10920 rpcz_store.cc:276] 0809 15:52:56.909584 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:56.913326 (+ 3742us) service_pool.cc:227] Handling call
0809 15:52:58.131514 (+1218188us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b540034f154. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478af0 00007f79694fbf2a 00007f796e438c6e 00007f7967b2f807 00007f7967b2f792 00007f7967b2f6c1 00007f7967b2f63c 00007f7967b2e8f2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:58.132037 (+ 523us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":420107520}
I20260809 15:52:58.150544 11014 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
I20260809 15:52:58.189097 14342 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:58.188223 13979 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:58.191072 10847 raft_consensus.cc:1226] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260809 15:52:58.154975 10811 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
W20260809 15:52:58.156353 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 44820.5us
I20260809 15:52:58.171060 14335 raft_consensus.cc:2781] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:58.124545 14383 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:58.131852 14325 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.122673 13913 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:58.107530 14329 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:58.068333 14409 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:58.198853 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.205003 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:58.178287 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.222702 10844 raft_consensus.cc:1226] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:58.212177 11640 raft_consensus.cc:2454] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:58.243949 14401 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.253895 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:58.254480 11640 raft_consensus.cc:3037] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:58.094398 14400 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:58.247104 13913 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:58.259847 14400 leader_election.cc:288] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:58.260288 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:58.260439 14400 raft_consensus.cc:482] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:58.261066 14400 raft_consensus.cc:504] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.249671 14329 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.264047 14329 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.266120 14409 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:58.268033 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:58.269098 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:58.154384 14261 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:58.133237 14405 leader_election.cc:288] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:52:58.239420 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 84568.5us
I20260809 15:52:58.240242 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:58.198012 14348 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 15:52:58.216262 14319 consensus_meta.cc:287] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.551s user 0.011s sys 0.000s
I20260809 15:52:58.192584 14267 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.198951 14133 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:58.210353 14252 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:58.225265 14331 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:58.238293 13925 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
W20260809 15:52:58.270169 10978 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.543s user 0.008s sys 0.032s
I20260809 15:52:58.239101 14335 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:58.272806 11640 raft_consensus.cc:2454] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:58.291148 10811 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:58.291813 10842 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:52:58.244063 14318 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:58.326225 10811 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:58.273074 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.327293 11642 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 8.
I20260809 15:52:58.220880 14312 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.246133 14094 raft_consensus.cc:2781] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:58.248457 14194 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.339365 14400 leader_election.cc:288] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.299198 14405 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:58.193876 14374 consensus_queue.cc:260] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.246416 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:58.300993 14133 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:52:58.355437 14261 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:58.329749 14329 raft_consensus.cc:2726] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:58.343650 14094 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:58.357121 14094 raft_consensus.cc:3037] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
W20260809 15:52:58.307322 10978 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.580s user 0.008s sys 0.032s
I20260809 15:52:58.307768 10841 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:58.254441 14401 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:58.365010 14133 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:58.302785 14421 log.cc:924] Time spent T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.241s user 0.000s sys 0.001s
I20260809 15:52:58.370800 14133 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.328120 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:58.376894 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:58.292815 11009 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:52:58.275192 10848 raft_consensus.cc:1261] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260809 15:52:58.329442 10811 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:58.331125 14391 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.331269 14048 raft_consensus.cc:2781] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260809 15:52:58.332530 14379 log.cc:924] Time spent T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 1.473s user 0.010s sys 0.011s
I20260809 15:52:58.327445 14335 raft_consensus.cc:3037] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:58.333001 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:58.297612 14030 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:58.347817 10921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54236 (request call id 2090) took 1424 ms. Trace:
I20260809 15:52:58.386698 10921 rpcz_store.cc:276] 0809 15:52:56.923802 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:56.923940 (+ 138us) service_pool.cc:227] Handling call
0809 15:52:57.583834 (+659894us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:57.658607 (+ 74773us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:58.342454 (+683847us) spinlock_profiling.cc:243] Waited 659 ms on lock 0x7b540025b7d0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:58.347739 (+ 5285us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1516770688}
I20260809 15:52:58.388159 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:58.389055 10921 raft_consensus.cc:2379] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 8: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
W20260809 15:52:58.329536 14424 log.cc:924] Time spent T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.186s user 0.001s sys 0.000s
W20260809 15:52:58.358724 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 631400us
I20260809 15:52:58.218533 14254 consensus_queue.cc:260] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.325660 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.308969 14252 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:58.358618 14405 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:58.373039 13913 raft_consensus.cc:2726] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:52:58.380712 14048 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:58.365269 14401 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:58.379251 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 49888.6us
I20260809 15:52:58.377641 11642 raft_consensus.cc:2362] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5. Re-sending same reply.
I20260809 15:52:58.413908 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:58.376262 14267 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:58.380105 14329 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:58.323994 14331 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.399461 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:58.403796 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:58.296300 14348 raft_consensus.cc:3037] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:58.419373 10921 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
I20260809 15:52:58.422673 10809 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:58.408924 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:58.409435 10978 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:58.410864 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.342502 14396 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.415335 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:58.417405 10811 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:58.364356 14368 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.420619 10842 raft_consensus.cc:1226] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 7 LEADER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260809 15:52:58.421456 10891 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:58.419592 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.423733 11010 raft_consensus.cc:1226] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
W20260809 15:52:58.530692 10923 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 2088) took 1610 ms (client timeout 1916 ms). Trace:
I20260809 15:52:58.412685 14048 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:58.431293 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:58.537081 10923 rpcz_store.cc:269] 0809 15:52:56.920728 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:56.920876 (+ 148us) service_pool.cc:227] Handling call
0809 15:52:58.530509 (+1609633us) spinlock_profiling.cc:243] Waited 865 ms on lock 0x7b5400373054. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:58.530631 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1990050176}
I20260809 15:52:58.545120 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:58.537154 11640 raft_consensus.cc:2454] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
W20260809 15:52:58.539345 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 122012us
W20260809 15:52:58.423889 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 221419us
I20260809 15:52:58.449427 10843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45578 (request call id 2067) took 1150 ms. Trace:
I20260809 15:52:58.452564 10840 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 7.
I20260809 15:52:58.439083 11636 raft_consensus.cc:1261] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:52:58.463261 10922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 2094) took 1536 ms. Trace:
I20260809 15:52:58.546715 10843 rpcz_store.cc:276] 0809 15:52:57.298584 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:57.298747 (+ 163us) service_pool.cc:227] Handling call
0809 15:52:57.313164 (+ 14417us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:57.667081 (+353917us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:58.449185 (+782104us) spinlock_profiling.cc:243] Waited 353 ms on lock 0x7b54001c5a54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:58.449347 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":811611520}
I20260809 15:52:58.428548 10844 raft_consensus.cc:1261] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 15:52:58.546928 11606 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:58.548904 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:52:58.549346 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 140014us
I20260809 15:52:58.510947 10929 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 11.
W20260809 15:52:58.545341 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 123951us
I20260809 15:52:58.549649 11010 raft_consensus.cc:2379] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 5: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:58.538029 10837 raft_consensus.cc:2454] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:58.547461 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.550022 10921 raft_consensus.cc:2396] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260809 15:52:58.550769 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.553030 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.424173 10920 raft_consensus.cc:2379] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 7: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:58.441113 11632 raft_consensus.cc:2396] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260809 15:52:58.552912 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:58.546396 10923 raft_consensus.cc:2379] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 7: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:58.555162 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:58.557384 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:58.567564 11639 raft_consensus.cc:2379] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 7: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:58.569963 11640 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 8.
I20260809 15:52:58.570644 11015 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:58.439996 14394 raft_consensus.cc:3032] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:52:58.430783 14261 raft_consensus.cc:2781] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:52:58.433502 14428 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:58.571326 10978 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:58.572463 10921 raft_consensus.cc:2396] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 9 because replica has last-logged OpId of term: 8 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 7.
I20260809 15:52:58.547427 10922 rpcz_store.cc:276] 0809 15:52:56.926857 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:56.927026 (+ 169us) service_pool.cc:227] Handling call
0809 15:52:58.463061 (+1536035us) spinlock_profiling.cc:243] Waited 1.11 s on lock 0x7b540025c6d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:58.463208 (+ 147us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2543881088}
I20260809 15:52:58.575402 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:58.445302 14405 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:58.381654 14049 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:58.445535 14262 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:58.578299 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:58.579033 11606 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:58.548717 13973 raft_consensus.cc:2781] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260809 15:52:58.438634 11633 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41488 (request call id 924) took 2546 ms (client timeout 1988 ms). Trace:
I20260809 15:52:58.444255 14252 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:58.448717 14331 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.424572 14400 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:58.582255 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.583123 14401 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.583300 14123 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:58.584198 10920 raft_consensus.cc:1226] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:58.553970 14280 raft_consensus.cc:2781] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:52:58.569767 14047 raft_consensus.cc:2781] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:58.586282 14432 raft_consensus.cc:2781] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:58.589519 10978 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:58.590269 14325 consensus_queue.cc:1035] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:58.593011 14329 raft_consensus.cc:2726] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:52:58.594449 14434 consensus_queue.cc:1035] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.001s
W20260809 15:52:58.593786 13958 consensus_meta.cc:287] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.575s user 0.006s sys 0.003s
I20260809 15:52:58.425191 14391 consensus_queue.cc:1046] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 5, Time since last communication: 0.160s
I20260809 15:52:58.461397 14306 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.411453 14094 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.632973 14047 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:58.607357 10922 raft_consensus.cc:1261] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 4. Preceding OpId from leader: term: 4 index: 4. (term mismatch)
I20260809 15:52:58.645862 10922 pending_rounds.cc:76] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3: Aborting all ops after (but not including) 3
I20260809 15:52:58.646250 10922 pending_rounds.cc:97] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3: Aborting uncommitted NO_OP operation due to leader change: 3.4
I20260809 15:52:58.646697 10922 raft_consensus.cc:2866] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:58.642603 14047 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:58.648861 14094 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.633258 14123 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:58.653955 14123 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:58.640630 14401 raft_consensus.cc:2781] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:58.654680 14401 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:58.654983 14401 raft_consensus.cc:3037] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
W20260809 15:52:58.629185 11633 rpcz_store.cc:269] 0809 15:52:55.892110 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:56.007110 (+115000us) service_pool.cc:227] Handling call
0809 15:52:58.438410 (+2431300us) spinlock_profiling.cc:243] Waited 380 ms on lock 0x7b54006bcbd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:58.438540 (+ 130us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":874647296}
I20260809 15:52:58.658776 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:58.659391 10920 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:52:58.660351 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:58.637315 14433 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:58.639658 14400 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:58.664928 14400 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:58.666283 14368 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:58.621539 13973 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:52:58.667732 13973 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:58.669814 14432 raft_consensus.cc:686] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:58.670814 14432 consensus_queue.cc:237] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:58.675204 11636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 950) took 2546 ms. Trace:
I20260809 15:52:58.675691 11636 rpcz_store.cc:276] 0809 15:52:56.128335 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:57.259117 (+1130782us) service_pool.cc:227] Handling call
0809 15:52:58.438774 (+1179657us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:58.547123 (+108349us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:58.674975 (+127852us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54006bcbd0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:58.675137 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2711692544}
I20260809 15:52:58.631453 13299 raft_consensus.cc:2726] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:58.633024 14280 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 15:52:58.621743 10893 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 0.639s user 0.023s sys 0.012s
I20260809 15:52:58.421948 14267 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.628614 11642 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:58.571098 13925 raft_consensus.cc:2726] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:52:58.530356 14188 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.631556 11013 raft_consensus.cc:1226] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 2. Current term is 3. Ops: []
I20260809 15:52:58.461539 14342 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:58.693203 14252 consensus_queue.cc:1046] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 5.3, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:52:58.694624 14342 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:58.581194 14261 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:58.576568 14394 raft_consensus.cc:729] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:58.631752 11604 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:58.637709 11014 raft_consensus.cc:1261] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 7 index: 5. (term mismatch)
I20260809 15:52:58.704136 11014 pending_rounds.cc:76] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa: Aborting all ops after (but not including) 4
I20260809 15:52:58.704386 11014 pending_rounds.cc:97] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260809 15:52:58.704624 11014 raft_consensus.cc:2866] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:52:58.704696 11011 raft_consensus.cc:1226] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
I20260809 15:52:58.638747 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:58.639147 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 49716.3us
I20260809 15:52:58.638000 10839 raft_consensus.cc:2379] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 6: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:58.608433 11015 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
I20260809 15:52:58.615093 11606 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:58.615371 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:58.707320 11018 raft_consensus.cc:2396] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
W20260809 15:52:58.707283 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 92258.8us
W20260809 15:52:58.707329 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 75648.5us
I20260809 15:52:58.683260 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:58.610251 14436 consensus_queue.cc:1035] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.065s
W20260809 15:52:58.685889 10893 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 0.703s user 0.023s sys 0.012s
I20260809 15:52:58.698989 10931 raft_consensus.cc:2379] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 6: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:58.702564 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:58.705875 10921 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
W20260809 15:52:58.710971 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 728451us
I20260809 15:52:58.708875 10922 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
I20260809 15:52:58.709996 10809 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:58.712301 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:58.710970 10978 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:58.713114 10921 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 8.
I20260809 15:52:58.713243 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:58.713559 10978 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:58.683218 10930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 2050) took 1223 ms. Trace:
I20260809 15:52:58.714268 10930 rpcz_store.cc:276] 0809 15:52:57.460103 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:57.468378 (+ 8275us) service_pool.cc:227] Handling call
0809 15:52:57.468686 (+ 308us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:57.655815 (+187129us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:57.655863 (+ 48us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:57.655906 (+ 43us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:57.655954 (+ 48us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:58.682931 (+1026977us) spinlock_profiling.cc:243] Waited 187 ms on lock 0x7b540015bdd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:58.682994 (+ 63us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:58.683147 (+ 153us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":429879168}
I20260809 15:52:58.715456 10978 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:58.716882 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:58.717722 10811 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:58.719075 10978 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:58.719569 10811 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:58.688781 14280 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:58.720955 10811 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:58.721834 10893 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:58.713903 10922 raft_consensus.cc:3037] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:58.718564 14306 raft_consensus.cc:2726] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:52:58.724395 14396 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.724045 14401 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:58.724742 14405 raft_consensus.cc:2726] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:58.724844 14388 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:58.725350 14213 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:58.725840 14172 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:52:58.727591 14267 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.728634 11606 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:58.729081 14094 raft_consensus.cc:2781] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:52:58.730124 14312 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.730826 10924 raft_consensus.cc:2379] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 7: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:58.731446 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:58.613219 14428 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.733028 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:58.741173 14329 raft_consensus.cc:2781] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 15:52:58.641335 14335 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:58.716439 14345 consensus_queue.cc:1035] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.037s
I20260809 15:52:58.735776 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:58.721879 10921 raft_consensus.cc:2396] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 7 because replica has last-logged OpId of term: 3 index: 2, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1.
I20260809 15:52:58.711453 14261 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:58.709713 13958 raft_consensus.cc:2726] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:58.743394 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:58.770398 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:58.770550 10837 raft_consensus.cc:2454] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:58.719533 14400 raft_consensus.cc:2781] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:58.778434 14400 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:58.778867 14400 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:58.780141 14312 consensus_queue.cc:1035] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:52:58.780471 10893 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:58.781153 11640 raft_consensus.cc:1261] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260809 15:52:58.781255 14342 consensus_queue.cc:1035] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260809 15:52:58.719718 14402 raft_consensus.cc:2781] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:52:58.782821 11640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 744) took 1088 ms. Trace:
I20260809 15:52:58.783265 14402 raft_consensus.cc:686] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 7 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:58.783331 11640 rpcz_store.cc:276] 0809 15:52:57.693969 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:58.582810 (+888841us) service_pool.cc:227] Handling call
0809 15:52:58.588762 (+ 5952us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:58.781610 (+192848us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:58.782518 (+ 908us) spinlock_profiling.cc:243] Waited 192 ms on lock 0x7b54007a0654. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:58.782749 (+ 231us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":441676544}
I20260809 15:52:58.784348 14402 consensus_queue.cc:237] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.787972 14254 raft_consensus.cc:3037] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 2 FOLLOWER]: Advancing to term 3
I20260809 15:52:58.788713 10893 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:58.720616 14297 raft_consensus.cc:2726] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 15:52:58.730665 14164 raft_consensus.cc:2781] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:52:58.755721 11606 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:58.794418 14164 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:58.794802 14164 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:58.797621 14220 raft_consensus.cc:3037] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:58.748198 10922 raft_consensus.cc:2454] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:52:58.728700 14438 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.029s
W20260809 15:52:58.758267 14415 log.cc:924] Time spent T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.936s user 0.002s sys 0.003s
I20260809 15:52:58.759070 11632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41482 (request call id 735) took 1067 ms. Trace:
I20260809 15:52:58.760046 14094 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:58.760938 11018 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
W20260809 15:52:58.771005 14390 log.cc:924] Time spent T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 1.523s user 0.004s sys 0.011s
I20260809 15:52:58.764612 14329 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:58.734329 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:58.719077 14252 consensus_queue.cc:1046] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.033s
I20260809 15:52:58.734958 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:58.826910 11010 raft_consensus.cc:2454] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:58.829892 11015 raft_consensus.cc:3037] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:58.768126 10811 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:52:58.831164 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 63116.6us
I20260809 15:52:58.833253 10811 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:58.772059 10978 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:58.763994 14030 consensus_queue.cc:1035] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.009s
I20260809 15:52:58.754844 14172 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:58.769156 13979 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:58.774739 10847 raft_consensus.cc:2379] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:58.775097 14335 leader_election.cc:288] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:58.781584 14312 raft_consensus.cc:2781] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:52:58.848004 14401 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:58.848097 14164 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.848670 14312 raft_consensus.cc:686] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:58.790350 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:58.849629 14312 consensus_queue.cc:237] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.850333 11009 raft_consensus.cc:3037] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:58.790652 11639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41488 (request call id 961) took 1075 ms. Trace:
I20260809 15:52:58.719956 14258 raft_consensus.cc:2781] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:58.792605 10929 raft_consensus.cc:1261] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 15:52:58.827232 14094 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:58.681272 14433 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.816146 10893 leader_election.cc:302] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:58.761464 11014 raft_consensus.cc:1226] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:58.824355 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:58.824805 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:58.825528 10839 raft_consensus.cc:1226] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 6. Current term is 7. Ops: [6.7-6.7]
I20260809 15:52:58.827044 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:58.864164 10842 raft_consensus.cc:2454] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:52:58.828843 10837 raft_consensus.cc:1261] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch)
I20260809 15:52:58.829100 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:58.834196 10891 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:52:58.834339 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 62367.3us
I20260809 15:52:58.834864 13958 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:58.835345 14223 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:58.754673 14213 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.842664 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.841779 14297 raft_consensus.cc:2781] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:58.818289 14192 raft_consensus.cc:2741] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260809 15:52:58.753479 14388 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.900403 14388 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.901051 14388 raft_consensus.cc:2726] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:58.903134 14331 raft_consensus.cc:2781] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:58.903575 14331 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:58.903915 14331 raft_consensus.cc:3037] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:58.909116 14428 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.844774 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:58.909458 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 75489.4us
I20260809 15:52:58.909984 11640 raft_consensus.cc:2454] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:58.910962 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:58.911535 11640 raft_consensus.cc:2362] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7. Re-sending same reply.
I20260809 15:52:58.912320 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.912853 11640 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:58.765350 14319 consensus_queue.cc:1046] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.061s
I20260809 15:52:58.914330 14319 raft_consensus.cc:3032] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:52:58.914737 14319 raft_consensus.cc:729] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:58.915645 14319 consensus_queue.cc:260] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.916200 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:58.916805 11013 raft_consensus.cc:2454] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:58.916836 14319 raft_consensus.cc:3037] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:58.859553 14431 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260809 15:52:58.917964 10891 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:58.816900 11632 rpcz_store.cc:276] 0809 15:52:57.691625 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:58.216422 (+524797us) service_pool.cc:227] Handling call
0809 15:52:58.758856 (+542434us) spinlock_profiling.cc:243] Waited 92.1 ms on lock 0x7b540008a054. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:58.759009 (+ 153us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":211766016}
I20260809 15:52:58.918726 11635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 947) took 2791 ms. Trace:
I20260809 15:52:58.841952 14123 raft_consensus.cc:2726] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:52:58.919245 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.853564 11639 rpcz_store.cc:276] 0809 15:52:57.715611 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:58.096068 (+380457us) service_pool.cc:227] Handling call
0809 15:52:58.790473 (+694405us) spinlock_profiling.cc:243] Waited 470 ms on lock 0x7b5400719dd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:58.790592 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1081669376}
I20260809 15:52:58.847455 11606 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:58.920933 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:58.762283 14219 raft_consensus.cc:2726] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:58.856284 14258 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:58.921600 14431 raft_consensus.cc:2726] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260809 15:52:58.921337 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 73958us
I20260809 15:52:58.923520 11606 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:58.923295 14261 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:58.925179 11606 leader_election.cc:302] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:58.924948 14047 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:58.830111 14329 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:52:58.756073 14394 consensus_queue.cc:260] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 5.3, Last appended by leader: 3, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:58.830542 14049 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:58.926035 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:58.926363 11012 raft_consensus.cc:2379] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 6: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:58.926818 14238 raft_consensus.cc:2726] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:58.927426 14238 raft_consensus.cc:2726] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority
I20260809 15:52:58.928145 14283 raft_consensus.cc:2781] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:58.928516 14283 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:58.928810 14283 raft_consensus.cc:3037] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:58.929361 14431 consensus_queue.cc:1035] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 15:52:58.931432 11606 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:58.931847 14172 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:58.933367 14238 raft_consensus.cc:2726] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:58.934036 14238 consensus_queue.cc:1046] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 15:52:58.934286 14348 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.935226 14192 raft_consensus.cc:3032] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:52:58.935534 11637 raft_consensus.cc:2379] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 5: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:58.935586 14192 raft_consensus.cc:729] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:58.936849 11637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41488 (request call id 955) took 1229 ms. Trace:
I20260809 15:52:58.936571 14192 consensus_queue.cc:260] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.937211 11637 rpcz_store.cc:276] 0809 15:52:57.707660 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:58.185637 (+477977us) service_pool.cc:227] Handling call
0809 15:52:58.936665 (+751028us) spinlock_profiling.cc:243] Waited 688 ms on lock 0x7b54007a1554. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:58.936799 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1581575424}
I20260809 15:52:58.938189 14192 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:58.939046 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:58.939477 14348 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:58.939782 11017 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:58.921226 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:58.943482 11639 raft_consensus.cc:3037] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:58.943912 11606 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:58.945662 14362 raft_consensus.cc:2726] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 15:52:58.854318 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:58.946568 11016 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:58.946866 11606 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:58.947981 11606 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:58.857604 14188 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.872704 10978 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:58.879937 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
W20260809 15:52:58.953639 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 81034.6us
I20260809 15:52:58.951758 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:58.955093 10978 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:58.882915 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:58.841923 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.842235 14332 raft_consensus.cc:2726] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:58.877038 14164 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.888924 14213 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:58.885037 14223 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.893180 10920 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 9.
I20260809 15:52:58.917250 13979 raft_consensus.cc:2726] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:58.886269 14297 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:58.921185 14331 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:58.875241 10922 raft_consensus.cc:2354] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for earlier term 5. Current term is 6.
I20260809 15:52:58.950765 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:58.948554 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:58.919139 11635 rpcz_store.cc:276] 0809 15:52:56.127446 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:56.847885 (+720439us) service_pool.cc:227] Handling call
0809 15:52:56.864535 (+ 16650us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:56.864782 (+ 247us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:56.864807 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:56.876786 (+ 11979us) log.cc:842] Serialized 26 byte log entry
0809 15:52:57.133456 (+256670us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:57.133507 (+ 51us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:57.133616 (+ 109us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:52:58.741390 (+1607774us) spinlock_profiling.cc:243] Waited 12.2 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:52:58.918422 (+177032us) raft_consensus.cc:1671] finished
0809 15:52:58.918447 (+ 25us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:58.918667 (+ 220us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":28094720,"thread_start_us":71573,"threads_started":1,"wal-append.queue_time_us":332821}
I20260809 15:52:58.861496 11009 raft_consensus.cc:2454] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:58.982254 10837 raft_consensus.cc:3037] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:58.983142 10809 leader_election.cc:302] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:58.952096 11632 raft_consensus.cc:2362] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 11 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 11. Re-sending same reply.
I20260809 15:52:58.847209 10811 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:58.875536 11015 raft_consensus.cc:2454] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:58.953678 10929 raft_consensus.cc:3037] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:58.884706 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:58.984790 10923 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:58.959659 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:58.960461 11636 raft_consensus.cc:2354] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7.
I20260809 15:52:58.957795 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:58.986335 10847 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 11.
I20260809 15:52:58.963196 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:58.923255 14433 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:52:58.991544 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 144465us
I20260809 15:52:58.992518 10811 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:58.952951 13925 raft_consensus.cc:3037] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:58.960781 14436 raft_consensus.cc:2726] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 15:52:58.967080 11018 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 7.
W20260809 15:52:58.969614 10978 leader_election.cc:334] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: RPC error from VoteRequest() call to peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783): Timed out: RequestConsensusVote RPC to 127.9.84.2:32783 timed out after 1.589s (SENT)
I20260809 15:52:58.946902 14394 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:58.921887 14258 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:58.883574 13958 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.008464 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:59.009128 11635 raft_consensus.cc:2454] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:58.980996 14342 raft_consensus.cc:2741] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Leader pre-election decision vote started in defunct term 8: won
I20260809 15:52:58.981237 14297 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:58.986374 11606 leader_election.cc:398] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Vote denied by peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783) with higher term. Message: Invalid argument: T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for earlier term 5. Current term is 6.
I20260809 15:52:58.954906 11637 raft_consensus.cc:2362] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6. Re-sending same reply.
I20260809 15:52:58.987175 11642 raft_consensus.cc:2454] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:58.987844 10891 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:58.988396 11639 raft_consensus.cc:2454] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:58.964298 14428 raft_consensus.cc:3037] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:58.960227 14261 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:58.996307 10811 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:58.997092 14123 raft_consensus.cc:2726] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 15:52:58.989293 14005 consensus_queue.cc:1035] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.008s
I20260809 15:52:58.987175 14362 raft_consensus.cc:2726] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:52:58.996199 11016 raft_consensus.cc:2454] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:52:58.986387 11640 raft_consensus.cc:2379] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 3: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
W20260809 15:52:59.004617 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 188549us
I20260809 15:52:59.007555 10929 raft_consensus.cc:2454] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:59.014899 11606 leader_election.cc:401] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260809 15:52:59.002629 14164 raft_consensus.cc:2781] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:58.990394 14332 raft_consensus.cc:2781] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:58.971407 13973 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:59.024583 10811 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:59.024856 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:59.025599 10923 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 7.
I20260809 15:52:59.036556 10891 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:59.022401 10837 raft_consensus.cc:2454] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:52:59.038419 10847 raft_consensus.cc:2379] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:59.039466 10978 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:59.039621 10811 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
W20260809 15:52:59.040050 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 53803.4us
I20260809 15:52:59.041546 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 11 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:59.041913 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:59.042316 11639 raft_consensus.cc:2379] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 11: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:59.042819 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:59.043541 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:59.044032 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:59.044389 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:59.045086 11639 raft_consensus.cc:2396] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 6 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
I20260809 15:52:59.045424 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:59.022894 14388 raft_consensus.cc:2781] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:59.045982 11642 raft_consensus.cc:2362] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 9. Re-sending same reply.
I20260809 15:52:59.048491 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:59.049239 11639 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:59.047379 10929 raft_consensus.cc:1226] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: [6.5-6.5]
I20260809 15:52:59.050135 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:59.051017 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:59.052335 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:59.053094 11642 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:52:59.050331 13806 raft_consensus.cc:2781] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:52:59.055286 10920 raft_consensus.cc:1261] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 15:52:59.056174 13958 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:59.056555 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:59.057307 10931 raft_consensus.cc:2379] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:59.057986 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:59.030452 14261 raft_consensus.cc:2726] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:59.058578 11637 raft_consensus.cc:2362] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Already granted yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5. Re-sending same reply.
I20260809 15:52:59.059512 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:59.060213 11637 raft_consensus.cc:2379] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 5: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:59.058478 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:59.064075 14329 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:59.066874 14331 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:59.067554 14213 raft_consensus.cc:2781] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:52:59.068087 14213 raft_consensus.cc:686] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:59.069005 14213 consensus_queue.cc:237] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.070240 14033 consensus_queue.cc:1035] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:59.051963 14436 raft_consensus.cc:2781] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:52:59.072813 14436 raft_consensus.cc:686] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:59.073441 10811 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:59.073639 14436 consensus_queue.cc:237] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:59.074004 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:59.084215 10840 raft_consensus.cc:1226] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:59.084537 11016 raft_consensus.cc:3037] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:59.060899 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:59.084942 10843 raft_consensus.cc:1226] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 8. Current term is 9. Ops: [6.5-8.6]
I20260809 15:52:59.075588 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:59.085613 10926 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:59.086153 14388 raft_consensus.cc:686] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 6 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:59.086686 13973 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:59.087196 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:59.087072 14388 consensus_queue.cc:237] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.088124 14216 consensus_queue.cc:260] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:59.089296 14216 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:59.091092 14047 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:59.091202 14164 raft_consensus.cc:686] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:59.052542 14300 consensus_queue.cc:1046] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.044s
I20260809 15:52:59.092147 14164 consensus_queue.cc:237] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.093302 13806 raft_consensus.cc:686] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:59.091104 14049 raft_consensus.cc:2781] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Leader election won for term 9
I20260809 15:52:59.095001 14049 raft_consensus.cc:686] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 9 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:59.095880 14049 consensus_queue.cc:237] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.047876 14332 raft_consensus.cc:686] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:59.097535 13925 raft_consensus.cc:2726] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:59.098388 14332 consensus_queue.cc:237] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 4.3, Last appended by leader: 3, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:59.100883 10927 raft_consensus.cc:2454] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:52:59.076538 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:59.104300 11640 raft_consensus.cc:2396] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 7 because replica has last-logged OpId of term: 6 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260809 15:52:59.101629 14455 consensus_queue.cc:1035] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.053s
I20260809 15:52:59.077335 11642 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 9.
I20260809 15:52:59.094473 13806 consensus_queue.cc:237] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 3.6, Last appended by leader: 6, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.107929 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:59.108059 14428 raft_consensus.cc:2726] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:59.108563 11635 raft_consensus.cc:2454] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
I20260809 15:52:59.083000 14400 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.110520 11013 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:59.047818 14345 raft_consensus.cc:2726] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 15:52:59.113552 10978 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:59.102334 11012 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:59.114758 10978 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:59.115281 11604 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:59.079677 10921 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:59.048517 14362 raft_consensus.cc:3037] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:59.010565 14048 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.120708 11639 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
I20260809 15:52:59.121606 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:59.122778 10847 raft_consensus.cc:1261] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:59.127595 10978 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:59.122890 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:59.128635 10978 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:59.126040 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:59.126648 13958 raft_consensus.cc:2781] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:59.130339 10923 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 8.
I20260809 15:52:59.129776 14223 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:59.131875 10811 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 11 election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:59.130707 10926 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 8.
I20260809 15:52:59.114647 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:59.136656 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:59.137388 10926 raft_consensus.cc:2379] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:59.133507 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:59.139297 10838 raft_consensus.cc:1226] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260809 15:52:59.139415 10929 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:59.141054 10929 raft_consensus.cc:1226] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260809 15:52:59.145251 11016 raft_consensus.cc:2454] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:59.130506 13958 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:59.109933 14335 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.037s
I20260809 15:52:59.134465 10978 leader_election.cc:398] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Vote denied by peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685) with higher term. Message: Invalid argument: T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7.
I20260809 15:52:59.151549 10891 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:59.153865 10811 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:59.147755 14401 raft_consensus.cc:2781] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:52:59.123783 14047 raft_consensus.cc:2726] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:59.161158 10848 raft_consensus.cc:2379] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 7: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:59.153887 11017 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 8.
I20260809 15:52:59.176900 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:59.177310 10809 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:59.160678 10978 leader_election.cc:401] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260809 15:52:59.123152 14252 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:59.159950 10811 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
W20260809 15:52:59.179199 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 44899.3us
I20260809 15:52:59.168073 10929 raft_consensus.cc:1261] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
I20260809 15:52:59.158188 13958 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:59.171093 11606 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:59.171875 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:59.179888 10978 leader_election.cc:302] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:59.180280 11639 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 11.
I20260809 15:52:59.172581 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:59.181227 11642 raft_consensus.cc:2454] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:59.173152 14223 consensus_queue.cc:1035] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:52:59.148180 14407 raft_consensus.cc:2781] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:59.174412 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:59.177907 10923 raft_consensus.cc:2379] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 5: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:59.173241 14258 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.182156 10978 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:52:59.182708 11606 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:52:59.185238 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:59.186044 10837 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:59.189370 11606 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:59.185448 10811 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:59.190825 10811 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:59.159622 14400 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:59.191634 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:59.158906 14428 raft_consensus.cc:2741] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election decision vote started in defunct term 5: won
I20260809 15:52:59.194569 10978 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:59.160341 13973 raft_consensus.cc:2726] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 11 FOLLOWER]: Leader election lost for term 11. Reason: could not achieve majority
I20260809 15:52:59.214473 10978 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:59.125272 14094 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:59.201210 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:59.166854 14402 raft_consensus.cc:2726] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 15:52:59.187103 14123 raft_consensus.cc:2781] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 15:52:59.181360 14401 raft_consensus.cc:686] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:59.187984 14405 raft_consensus.cc:2726] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685) with higher term. Message: Invalid argument: T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7.
I20260809 15:52:59.185775 14407 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:52:59.192119 14005 raft_consensus.cc:2781] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:52:59.193925 14306 raft_consensus.cc:2781] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:52:59.179708 14331 raft_consensus.cc:2781] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:52:59.193562 14033 raft_consensus.cc:2726] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:59.197417 14348 raft_consensus.cc:2781] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:59.191979 14261 raft_consensus.cc:2781] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Leader election won for term 9
I20260809 15:52:59.179519 14254 consensus_queue.cc:1035] T 8975b3e7ddc544bc888dd61fffc5313b P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.054s
I20260809 15:52:59.187604 11634 raft_consensus.cc:2454] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:52:59.197918 14434 raft_consensus.cc:2741] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won
I20260809 15:52:59.180174 13925 consensus_queue.cc:1046] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.048s
I20260809 15:52:59.197187 14431 raft_consensus.cc:2726] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:52:59.214440 14332 raft_consensus.cc:2726] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:52:59.246809 10844 raft_consensus.cc:1226] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260809 15:52:59.230258 14400 raft_consensus.cc:2781] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:59.229374 14047 raft_consensus.cc:2741] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won
I20260809 15:52:59.185505 14252 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:59.229557 14455 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:59.200011 14030 consensus_queue.cc:1035] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.047s
I20260809 15:52:59.244088 14407 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:59.264125 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:59.247584 14306 raft_consensus.cc:482] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:59.254525 14123 raft_consensus.cc:686] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 8 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
W20260809 15:52:59.271112 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 56727us
I20260809 15:52:59.270947 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:59.256417 14005 raft_consensus.cc:686] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 7 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:52:59.261371 14331 raft_consensus.cc:686] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [term 6 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:59.268388 14261 raft_consensus.cc:686] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 9 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:59.261698 14348 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:59.240641 14280 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:59.286954 14004 consensus_queue.cc:1046] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.020s
I20260809 15:52:59.265100 14329 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:59.290381 10978 leader_election.cc:302] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 3 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:59.275223 14400 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:59.260775 14283 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.295780 14005 consensus_queue.cc:237] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.289670 14306 raft_consensus.cc:3037] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:52:59.296425 14331 consensus_queue.cc:237] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.297317 14123 consensus_queue.cc:237] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:59.287184 14401 consensus_queue.cc:237] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.304823 14258 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:59.326364 10978 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:59.309701 10893 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:59.327045 14400 raft_consensus.cc:3037] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:59.360138 10931 raft_consensus.cc:2379] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 8: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
W20260809 15:52:59.360556 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 50977.5us
I20260809 15:52:59.346563 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:59.332683 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:59.362200 10893 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:59.362625 11017 raft_consensus.cc:2379] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 7: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:59.342504 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:59.363504 11016 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:59.364033 10893 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:59.348296 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:52:59.362283 10838 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:59.370011 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:59.370179 10893 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:59.372412 10923 raft_consensus.cc:2379] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:59.372793 10978 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:59.310694 14348 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:59.377108 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
W20260809 15:52:59.382532 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 191760us
I20260809 15:52:59.387281 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:59.345842 14450 log.cc:924] Time spent T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.216s user 0.003s sys 0.000s
I20260809 15:52:59.388186 10811 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:59.390551 10847 raft_consensus.cc:1261] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 15:52:59.290978 14455 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.328083 14047 raft_consensus.cc:2726] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader election lost for term 3. Reason: could not achieve majority
I20260809 15:52:59.398278 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:59.400188 11016 raft_consensus.cc:2454] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:52:59.340056 14048 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:59.339928 14094 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:59.325577 14188 consensus_queue.cc:1046] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.062s
I20260809 15:52:59.406919 10811 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:59.406064 11606 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:52:59.340795 14252 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:59.406589 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:59.305995 14297 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:59.385097 14405 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 11 FOLLOWER]: Leader pre-election won for term 12
I20260809 15:52:59.387281 14434 raft_consensus.cc:2741] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260809 15:52:59.386976 14216 raft_consensus.cc:2781] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 15:52:59.374583 14382 raft_consensus.cc:2781] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:52:59.445076 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:52:59.451282 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 44412.2us
I20260809 15:52:59.371963 14172 raft_consensus.cc:2741] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260809 15:52:59.458767 10838 raft_consensus.cc:2454] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
W20260809 15:52:59.439654 14457 log.cc:924] Time spent T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.170s user 0.002s sys 0.000s
I20260809 15:52:59.480671 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:59.477921 10811 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:52:59.482272 10926 raft_consensus.cc:2379] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:52:59.486923 11011 raft_consensus.cc:2379] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 7: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:59.486729 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:59.409216 14033 raft_consensus.cc:2726] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 15:52:59.489353 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:59.481560 14382 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:59.460590 14296 raft_consensus.cc:2781] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election won for term 8
W20260809 15:52:59.525681 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 155591us
I20260809 15:52:59.519763 11604 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:52:59.527468 10893 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:59.490963 11016 raft_consensus.cc:1261] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260809 15:52:59.491734 14300 raft_consensus.cc:2781] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:59.495306 14455 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:59.495935 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:59.477166 14261 consensus_queue.cc:237] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.493821 11633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 755) took 1795 ms. Trace:
I20260809 15:52:59.506505 14297 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:52:59.508744 14462 log.cc:924] Time spent T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.104s user 0.000s sys 0.000s
I20260809 15:52:59.509907 10924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52758 (request call id 825) took 1666 ms. Trace:
W20260809 15:52:59.571517 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 165543us
I20260809 15:52:59.572842 10924 rpcz_store.cc:276] 0809 15:52:57.844002 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:57.844222 (+ 220us) service_pool.cc:227] Handling call
0809 15:52:59.509722 (+1665500us) spinlock_profiling.cc:243] Waited 736 ms on lock 0x7b540018b354. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:59.509850 (+ 128us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1692225792}
I20260809 15:52:59.574451 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:59.510123 14280 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:59.510766 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:59.580394 11606 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:59.511637 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:59.514972 14374 raft_consensus.cc:3037] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:59.458184 14405 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 11 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 15:52:59.475739 14465 log.cc:924] Time spent T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.125s user 0.002s sys 0.000s
I20260809 15:52:59.473942 14252 raft_consensus.cc:2741] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260809 15:52:59.478188 14192 raft_consensus.cc:2726] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:59.556731 10893 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:59.405462 14325 consensus_queue.cc:1035] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.119s
I20260809 15:52:59.460345 14047 consensus_queue.cc:1046] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.072s
W20260809 15:52:59.569113 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 120868us
I20260809 15:52:59.597292 14047 raft_consensus.cc:3032] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 LEADER]: Stepping down as leader of term 8
I20260809 15:52:59.597762 14047 raft_consensus.cc:729] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:52:59.572238 14094 consensus_queue.cc:1035] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260809 15:52:59.530448 14382 raft_consensus.cc:3037] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:52:59.579085 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:59.579543 11637 raft_consensus.cc:1226] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 6. Current term is 7. Ops: [4.6-6.7]
I20260809 15:52:59.557749 14030 raft_consensus.cc:2781] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election won for term 6
W20260809 15:52:59.566004 14456 log.cc:924] Time spent T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.082s user 0.001s sys 0.000s
I20260809 15:52:59.580435 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:59.557798 13979 raft_consensus.cc:2726] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:52:59.581926 11018 raft_consensus.cc:2379] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 8: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:59.529825 14300 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:59.585346 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:59.595857 10926 raft_consensus.cc:2379] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 7: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:52:59.568055 11633 rpcz_store.cc:276] 0809 15:52:57.699016 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:58.658872 (+959856us) service_pool.cc:227] Handling call
0809 15:52:58.659187 (+ 315us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:58.718547 (+ 59360us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:58.730744 (+ 12197us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:58.736156 (+ 5412us) raft_consensus.cc:1631] Marking committed up to 7
0809 15:52:58.740158 (+ 4002us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:59.493522 (+753364us) spinlock_profiling.cc:243] Waited 44.3 ms on lock 0x7b5400631854. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:52:59.493576 (+ 54us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:52:59.493755 (+ 179us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":101801856}
I20260809 15:52:59.584050 14405 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 11 FOLLOWER]: Advancing to term 12
W20260809 15:52:59.629994 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 73374.5us
I20260809 15:52:59.635637 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:59.643435 10838 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 7
I20260809 15:52:59.587308 14283 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:59.633989 10924 raft_consensus.cc:2362] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 8. Re-sending same reply.
I20260809 15:52:59.653472 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:59.600145 14280 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:52:59.654739 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:59.632337 14252 raft_consensus.cc:2726] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:59.569483 14296 raft_consensus.cc:686] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 8 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:59.663714 14296 consensus_queue.cc:237] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.647678 10893 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:52:59.632087 14433 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
W20260809 15:52:59.639364 14319 consensus_meta.cc:287] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.722s user 0.012s sys 0.003s
I20260809 15:52:59.670974 14319 raft_consensus.cc:2781] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:52:59.671643 14319 raft_consensus.cc:686] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:59.672725 14319 consensus_queue.cc:237] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.668057 10893 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:59.675414 14252 raft_consensus.cc:2781] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:52:59.675930 14252 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:59.676373 14252 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
W20260809 15:52:59.679167 14471 log.cc:924] Time spent T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.107s user 0.001s sys 0.000s
I20260809 15:52:59.659613 10924 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:52:59.659771 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:59.683276 10923 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:52:59.632522 14164 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:59.636096 14300 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:59.685753 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:52:59.646005 14394 consensus_meta.cc:287] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.638s user 0.011s sys 0.000s
I20260809 15:52:59.696177 11016 raft_consensus.cc:1261] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 15:52:59.697536 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:59.700091 10837 raft_consensus.cc:1226] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 6. Current term is 7. Ops: []
I20260809 15:52:59.702538 10843 raft_consensus.cc:1226] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260809 15:52:59.643491 14030 raft_consensus.cc:686] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
W20260809 15:52:59.707953 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 68285.4us
I20260809 15:52:59.715963 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:59.639065 14400 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.716728 11636 raft_consensus.cc:2362] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3. Re-sending same reply.
I20260809 15:52:59.722474 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:59.722637 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:59.724164 10922 raft_consensus.cc:2454] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:52:59.705894 10838 raft_consensus.cc:2454] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:52:59.725833 10847 raft_consensus.cc:1226] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 7. Ops: []
I20260809 15:52:59.726122 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:59.727936 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:59.726691 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:59.719696 10923 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 9.
I20260809 15:52:59.720017 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:59.733441 10837 raft_consensus.cc:2379] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 6: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:59.736075 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:59.736913 10837 raft_consensus.cc:2379] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 7: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:59.738184 10811 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:59.734481 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:59.706403 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:59.763548 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 3 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:59.636637 14047 consensus_queue.cc:260] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:59.766685 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:59.772204 10839 raft_consensus.cc:2454] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:59.767010 11637 raft_consensus.cc:2362] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3. Re-sending same reply.
I20260809 15:52:59.769663 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:59.770797 11017 raft_consensus.cc:2379] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 6: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:52:59.771194 11016 raft_consensus.cc:1261] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260809 15:52:59.776126 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:59.777019 10923 raft_consensus.cc:2454] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 3.
I20260809 15:52:59.776065 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:59.788714 14348 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.790388 11639 raft_consensus.cc:2354] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7.
I20260809 15:52:59.792984 14348 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:59.793491 14394 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:59.794523 14394 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.795419 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:52:59.797240 14394 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:52:59.797233 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:59.801654 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:52:59.800024 10893 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:59.803481 14394 raft_consensus.cc:2726] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:52:59.806715 14394 consensus_queue.cc:1046] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:52:59.808588 14455 raft_consensus.cc:3032] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:52:59.690532 14280 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.781114 10924 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 8.
I20260809 15:52:59.810101 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:59.811632 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:52:59.817926 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:59.818758 10924 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 11.
I20260809 15:52:59.819734 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:59.812269 10893 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:59.732566 14454 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:52:59.830773 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 250441us
I20260809 15:52:59.829329 10893 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:59.827414 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:59.828172 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:59.829255 11009 raft_consensus.cc:1261] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260809 15:52:59.721588 14094 consensus_queue.cc:1035] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.092s
I20260809 15:52:59.839711 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:59.840360 11636 raft_consensus.cc:2454] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:52:59.836638 10893 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:52:59.837198 10923 raft_consensus.cc:2379] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:52:59.842675 10893 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:59.814306 14262 raft_consensus.cc:2781] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Leader election won for term 9
I20260809 15:52:59.809062 14455 raft_consensus.cc:729] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:52:59.811496 11633 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 9.
I20260809 15:52:59.703428 14164 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.847978 14473 raft_consensus.cc:2781] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:52:59.849639 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:52:59.831703 13958 consensus_meta.cc:287] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.636s user 0.011s sys 0.003s
I20260809 15:52:59.844086 10893 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:52:59.852686 14030 consensus_queue.cc:237] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:52:59.869649 14480 log.cc:924] Time spent T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.142s user 0.000s sys 0.001s
W20260809 15:52:59.870431 14474 log.cc:924] Time spent T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.151s user 0.001s sys 0.000s
I20260809 15:52:59.839982 14172 raft_consensus.cc:2726] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 15:52:59.873076 10848 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 822) took 1062 ms. Trace:
I20260809 15:52:59.884317 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:59.897476 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:59.871951 14473 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:52:59.932829 10923 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:52:59.935374 11636 raft_consensus.cc:3037] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:59.840749 14394 raft_consensus.cc:2726] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:52:59.936209 10838 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:52:59.835533 14283 consensus_queue.cc:1035] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.071s
I20260809 15:52:59.854270 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:52:59.936846 10848 rpcz_store.cc:276] 0809 15:52:58.810774 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:58.810890 (+ 116us) service_pool.cc:227] Handling call
0809 15:52:59.872868 (+1061978us) spinlock_profiling.cc:243] Waited 294 ms on lock 0x7b54001edf54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:52:59.873015 (+ 147us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":677312384}
I20260809 15:52:59.948422 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:52:59.949167 11637 raft_consensus.cc:3037] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:59.951800 14004 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:52:59.901711 13958 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.952324 14004 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.879132 14262 raft_consensus.cc:686] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 9 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:52:59.852375 13979 consensus_queue.cc:1035] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.084s
I20260809 15:52:59.838718 14454 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:52:59.954681 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:52:59.937474 14473 raft_consensus.cc:3037] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:52:59.955495 11633 raft_consensus.cc:2454] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:52:59.964627 14262 consensus_queue.cc:237] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:52:59.969934 13958 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:52:59.974726 14004 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:52:59.936215 14319 raft_consensus.cc:2726] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 15:52:59.984155 10923 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 7.
W20260809 15:52:59.989539 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 251437us
I20260809 15:52:59.992238 11637 raft_consensus.cc:2454] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:52:59.905125 14252 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:52:59.997452 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 399214us
I20260809 15:52:59.967783 14394 raft_consensus.cc:2726] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:00.001266 11637 raft_consensus.cc:1226] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 4. Current term is 5. Ops: [3.6-4.7]
I20260809 15:52:59.934293 14300 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:00.020609 10931 raft_consensus.cc:1226] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 8. Current term is 9. Ops: []
I20260809 15:53:00.003001 10920 raft_consensus.cc:2379] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:00.025605 10837 raft_consensus.cc:1261] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260809 15:53:00.031106 10811 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:53:00.034890 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 123211us
I20260809 15:53:00.035665 10848 raft_consensus.cc:1261] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
W20260809 15:53:00.003839 14188 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.525s user 0.000s sys 0.023s
I20260809 15:53:00.037827 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:00.041280 11010 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:00.043599 10839 raft_consensus.cc:2454] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:53:00.045820 10978 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:00.049209 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:00.053962 10811 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:00.052142 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:52:59.972790 14455 consensus_queue.cc:260] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:00.073967 10922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 2098) took 1031 ms. Trace:
I20260809 15:53:00.081215 10922 rpcz_store.cc:276] 0809 15:52:59.042539 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:59.043311 (+ 772us) service_pool.cc:227] Handling call
0809 15:53:00.073809 (+1030498us) spinlock_profiling.cc:243] Waited 671 ms on lock 0x7b540018a954. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:00.073911 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1542261120}
I20260809 15:53:00.084695 11010 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 7.
I20260809 15:53:00.086335 11604 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:00.066720 10930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 2065) took 1485 ms. Trace:
I20260809 15:53:00.087558 10930 rpcz_store.cc:276] 0809 15:52:58.580909 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:58.771242 (+190333us) service_pool.cc:227] Handling call
0809 15:52:58.771523 (+ 281us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:58.874119 (+102596us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:58.874160 (+ 41us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:52:58.874210 (+ 50us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:58.874247 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:00.066386 (+1192139us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b5400371254. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:00.066449 (+ 63us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:00.066649 (+ 200us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":235390848}
I20260809 15:53:00.089936 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:53:00.046207 14188 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.567s user 0.001s sys 0.023s
I20260809 15:53:00.044775 14048 raft_consensus.cc:2726] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 15:53:00.131713 11606 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:00.140244 10930 raft_consensus.cc:1261] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260809 15:53:00.144516 10930 raft_consensus.cc:1261] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 15:53:00.045898 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:00.054242 14297 raft_consensus.cc:2781] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:53:00.149000 10930 raft_consensus.cc:1261] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 15:53:00.153762 10930 raft_consensus.cc:3032] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:53:00.154155 10930 raft_consensus.cc:729] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:00.154458 14030 raft_consensus.cc:2781] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election won for term 7
W20260809 15:53:00.056751 14306 consensus_meta.cc:287] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.687s user 0.010s sys 0.001s
W20260809 15:53:00.141193 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 54963.7us
I20260809 15:53:00.056267 14280 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:00.038936 13973 consensus_queue.cc:1046] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.101s
I20260809 15:53:00.155000 10930 consensus_queue.cc:260] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:00.236963 10930 raft_consensus.cc:3037] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:53:00.046239 14400 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:00.173105 14306 raft_consensus.cc:504] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:00.242405 14400 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:00.243041 14400 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:00.245023 14306 leader_election.cc:288] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:00.151288 14402 raft_consensus.cc:2781] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 15:53:00.154806 14247 raft_consensus.cc:3032] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:53:00.090230 14325 consensus_queue.cc:1035] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.082s
I20260809 15:53:00.262961 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:00.265352 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:53:00.271821 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 220576us
I20260809 15:53:00.273684 10841 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:00.273684 10837 raft_consensus.cc:2454] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
W20260809 15:53:00.280129 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 436065us
I20260809 15:53:00.161428 10920 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:53:00.283212 10841 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 7.
I20260809 15:53:00.235075 14297 raft_consensus.cc:686] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
W20260809 15:53:00.291010 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 237113us
I20260809 15:53:00.286057 10922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:54216 (request call id 1593) took 1171 ms. Trace:
I20260809 15:53:00.305797 10811 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:00.270331 14252 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:00.323807 10893 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:00.287328 14402 raft_consensus.cc:686] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 8 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:00.303946 10922 rpcz_store.cc:276] 0809 15:52:59.114228 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:00.284809 (+1170581us) service_pool.cc:227] Handling call
0809 15:53:00.285998 (+ 1189us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:53:00.261842 14247 raft_consensus.cc:729] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:00.374413 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:00.389643 10922 raft_consensus.cc:2379] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
W20260809 15:53:00.375538 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 69823.1us
W20260809 15:53:00.377458 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 89313.5us
I20260809 15:53:00.321493 14188 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:00.422997 10978 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:53:00.300201 14500 raft_consensus.cc:2781] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election won for term 7
W20260809 15:53:00.346578 14488 log.cc:924] Time spent T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.384s user 0.001s sys 0.000s
W20260809 15:53:00.346344 14489 log.cc:924] Time spent T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.379s user 0.000s sys 0.001s
I20260809 15:53:00.453945 11014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2441) took 1720 ms. Trace:
I20260809 15:53:00.349596 14498 raft_consensus.cc:2726] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:00.312323 10920 raft_consensus.cc:595] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:00.311196 14454 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:00.460669 11014 rpcz_store.cc:276] 0809 15:52:58.733085 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:58.733947 (+ 862us) service_pool.cc:227] Handling call
0809 15:52:58.749203 (+ 15256us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:58.860954 (+111751us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:00.453685 (+1592731us) spinlock_profiling.cc:243] Waited 12 ms on lock 0x7b540040b354. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:00.453885 (+ 200us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":27626368}
W20260809 15:53:00.454639 10927 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54224 (request call id 2058) took 2552 ms (client timeout 1637 ms). Trace:
I20260809 15:53:00.349417 14164 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:00.348912 14252 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
W20260809 15:53:00.477141 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 85828.1us
I20260809 15:53:00.366255 14400 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:00.473934 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 202079us
W20260809 15:53:00.467491 11605 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.566s user 0.002s sys 0.007s
W20260809 15:53:00.470969 10927 rpcz_store.cc:269] 0809 15:52:57.902248 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:57.902402 (+ 154us) service_pool.cc:227] Handling call
0809 15:53:00.454457 (+2552055us) spinlock_profiling.cc:243] Waited 1.1 s on lock 0x7b5400372b54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:00.454563 (+ 106us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2539942656}
I20260809 15:53:00.440896 11636 raft_consensus.cc:2454] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:53:00.496424 10838 raft_consensus.cc:1261] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260809 15:53:00.499643 14374 consensus_meta.cc:287] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.911s user 0.011s sys 0.000s
I20260809 15:53:00.500270 11641 raft_consensus.cc:2454] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
W20260809 15:53:00.502200 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 178487us
I20260809 15:53:00.503041 10893 leader_election.cc:302] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:00.504549 10893 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:00.455221 14300 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:00.495015 11605 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.594s user 0.002s sys 0.007s
I20260809 15:53:00.453428 14030 raft_consensus.cc:686] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 7 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:53:00.430209 14505 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:53:00.474119 14490 log.cc:924] Time spent T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.452s user 0.002s sys 0.000s
I20260809 15:53:00.510798 10844 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45578 (request call id 2082) took 1774 ms. Trace:
I20260809 15:53:00.519001 10844 rpcz_store.cc:276] 0809 15:52:58.736104 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:58.736210 (+ 106us) service_pool.cc:227] Handling call
0809 15:52:58.736493 (+ 283us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:59.269844 (+533351us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:00.509946 (+1240102us) spinlock_profiling.cc:243] Waited 507 ms on lock 0x7b54001c5a54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:00.510720 (+ 774us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1164967296}
I20260809 15:53:00.424791 14306 consensus_queue.cc:1035] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.061s
I20260809 15:53:00.514914 11642 raft_consensus.cc:1261] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260809 15:53:00.467955 14507 raft_consensus.cc:2726] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:53:00.478669 14500 raft_consensus.cc:686] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [term 7 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:53:00.502354 14192 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:00.456120 14188 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:00.460614 14325 consensus_queue.cc:1035] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.055s
W20260809 15:53:00.505614 14362 consensus_meta.cc:287] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 1.379s user 0.010s sys 0.007s
I20260809 15:53:00.466444 10920 raft_consensus.cc:603] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 15:53:00.532959 10893 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:00.532646 11642 raft_consensus.cc:1261] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260809 15:53:00.509649 14164 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:53:00.563903 14164 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:53:00.565218 14405 consensus_meta.cc:287] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.882s user 0.011s sys 0.000s
I20260809 15:53:00.512444 14498 raft_consensus.cc:2781] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 15:53:00.560158 14319 raft_consensus.cc:2726] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:53:00.519336 14402 consensus_queue.cc:237] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:00.509931 14454 consensus_queue.cc:1035] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.046s
I20260809 15:53:00.523975 14495 raft_consensus.cc:2726] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:53:00.562354 14500 consensus_queue.cc:237] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:00.563339 14518 raft_consensus.cc:482] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:00.559537 10893 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:53:00.592661 10930 raft_consensus.cc:1261] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 5 index: 7. (term mismatch)
I20260809 15:53:00.502552 14252 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:00.558584 14192 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:00.521574 14261 consensus_queue.cc:1035] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.082s
I20260809 15:53:00.517997 14297 consensus_queue.cc:237] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:00.523098 14247 consensus_queue.cc:260] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:00.542114 14505 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:00.657029 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 97586.7us
I20260809 15:53:00.812314 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:00.848649 10893 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
W20260809 15:53:00.764477 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 341593us
I20260809 15:53:00.849172 10927 raft_consensus.cc:2396] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260809 15:53:00.724275 14306 raft_consensus.cc:2781] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260809 15:53:00.762568 14494 log.cc:924] Time spent T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.502s user 0.002s sys 0.000s
I20260809 15:53:00.810978 14188 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:53:00.763204 14504 log.cc:924] Time spent T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.247s user 0.001s sys 0.000s
W20260809 15:53:00.764101 14509 log.cc:924] Time spent T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.087s user 0.000s sys 0.001s
W20260809 15:53:00.791221 14524 log.cc:924] Time spent T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.114s user 0.000s sys 0.000s
I20260809 15:53:00.883014 14521 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:53:00.763967 14407 consensus_meta.cc:287] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 1.478s user 0.030s sys 0.040s
I20260809 15:53:00.914516 14407 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:00.840922 13299 raft_consensus.cc:3032] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 15:53:00.912190 11633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 975) took 1178 ms. Trace:
I20260809 15:53:00.918320 13299 raft_consensus.cc:729] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:53:00.919857 13299 consensus_queue.cc:260] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:00.918561 11633 rpcz_store.cc:276] 0809 15:52:59.733283 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:00.294497 (+561214us) service_pool.cc:227] Handling call
0809 15:53:00.429009 (+134512us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:00.467991 (+ 38982us) raft_consensus.cc:1520] Early marking committed up to index 6
0809 15:53:00.548191 (+ 80200us) log.cc:842] Serialized 14 byte log entry
0809 15:53:00.830523 (+282332us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:00.851210 (+ 20687us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:53:00.897442 (+ 46232us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:00.901669 (+ 4227us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:00.910587 (+ 8918us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":23803,"prepare.queue_time_us":47826,"prepare.run_cpu_time_us":59,"prepare.run_wall_time_us":8650,"raft.queue_time_us":38802,"raft.run_cpu_time_us":90,"raft.run_wall_time_us":91,"thread_start_us":188543,"threads_started":1,"wal-append.queue_time_us":292866}
I20260809 15:53:00.654973 14498 raft_consensus.cc:686] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:00.892812 10978 leader_election.cc:302] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
W20260809 15:53:00.790431 14473 consensus_meta.cc:287] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.835s user 0.007s sys 0.005s
I20260809 15:53:00.708082 14519 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
W20260809 15:53:00.811111 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 254591us
I20260809 15:53:00.569474 14362 raft_consensus.cc:2726] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783) with higher term. Message: Invalid argument: T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for earlier term 5. Current term is 6.
I20260809 15:53:00.780812 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:00.684741 14495 raft_consensus.cc:2781] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
W20260809 15:53:00.833297 14433 consensus_meta.cc:287] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 1.162s user 0.015s sys 0.000s
I20260809 15:53:00.823124 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 853) took 1247 ms. Trace:
I20260809 15:53:00.682888 14030 consensus_queue.cc:237] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:00.887516 14252 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:53:00.842517 14382 consensus_meta.cc:287] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 1.208s user 0.009s sys 0.004s
W20260809 15:53:00.816114 14512 log.cc:924] Time spent T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.395s user 0.001s sys 0.000s
I20260809 15:53:00.852351 10930 pending_rounds.cc:76] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3: Aborting all ops after (but not including) 6
I20260809 15:53:00.948570 10930 pending_rounds.cc:97] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3: Aborting uncommitted NO_OP operation due to leader change: 4.7
I20260809 15:53:00.949069 10930 raft_consensus.cc:2866] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:53:00.950536 10930 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "0f891fe62df749108fa553cb0661a874"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:53:00.951184 10930 raft_consensus.cc:595] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:53:00.723316 14501 log.cc:924] Time spent T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.348s user 0.000s sys 0.001s
W20260809 15:53:00.723369 14502 log.cc:924] Time spent T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.261s user 0.000s sys 0.001s
W20260809 15:53:00.745770 14497 log.cc:924] Time spent T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.435s user 0.000s sys 0.001s
I20260809 15:53:00.831744 14499 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:53:00.908331 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 59566.5us
I20260809 15:53:00.780664 14517 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:00.711721 14503 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:00.695307 14319 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:00.911054 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:00.659724 14094 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:00.913596 14188 raft_consensus.cc:2781] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:00.917589 14529 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:00.980314 10927 raft_consensus.cc:2379] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 7: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:00.921679 13299 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:00.914000 14521 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:00.901747 14306 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:00.930899 10978 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:00.906136 14164 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:00.938683 10931 rpcz_store.cc:276] 0809 15:52:59.576062 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:00.079871 (+503809us) service_pool.cc:227] Handling call
0809 15:53:00.158041 (+ 78170us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:00.158302 (+ 261us) raft_consensus.cc:1520] Early marking committed up to index 8
0809 15:53:00.587857 (+429555us) log.cc:842] Serialized 14 byte log entry
0809 15:53:00.822811 (+234954us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:00.822857 (+ 46us) raft_consensus.cc:1631] Marking committed up to 8
0809 15:53:00.822901 (+ 44us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:00.822970 (+ 69us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:00.823068 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":22856,"prepare.queue_time_us":317153,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":47,"raft.queue_time_us":1124,"raft.run_cpu_time_us":258,"raft.run_wall_time_us":18542,"thread_start_us":662023,"threads_started":2,"wal-append.queue_time_us":283943}
I20260809 15:53:00.912878 14505 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:00.565753 14405 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 12 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:00.946849 11632 raft_consensus.cc:2454] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:53:00.953879 11018 raft_consensus.cc:1261] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260809 15:53:00.903019 14518 raft_consensus.cc:504] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:00.927861 14519 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:00.927784 14473 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.026429 11637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 972) took 1309 ms. Trace:
I20260809 15:53:00.985976 14188 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:01.026919 11637 rpcz_store.cc:276] 0809 15:52:59.717110 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:00.005547 (+288437us) service_pool.cc:227] Handling call
0809 15:53:00.073325 (+ 67778us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:00.073708 (+ 383us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:00.073738 (+ 30us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:00.089591 (+ 15853us) log.cc:842] Serialized 26 byte log entry
0809 15:53:00.250880 (+161289us) raft_consensus.cc:1631] Marking committed up to 7
0809 15:53:00.251480 (+ 600us) log.cc:842] Serialized 14 byte log entry
0809 15:53:00.426285 (+174805us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:00.426384 (+ 99us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:01.026190 (+599806us) raft_consensus.cc:1671] finished
0809 15:53:01.026207 (+ 17us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:01.026358 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":45458,"prepare.queue_time_us":140,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":42,"raft.queue_time_us":233568,"raft.run_cpu_time_us":301,"raft.run_wall_time_us":63105,"thread_start_us":228061,"threads_started":2,"wal-append.queue_time_us":169475}
W20260809 15:53:00.960022 10975 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.706s user 0.007s sys 0.016s
I20260809 15:53:00.936327 14495 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:00.974306 10893 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
W20260809 15:53:01.034837 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 60650us
I20260809 15:53:01.036127 10842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45578 (request call id 2085) took 2161 ms. Trace:
I20260809 15:53:01.036634 14521 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:00.943230 14433 raft_consensus.cc:2726] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:00.943579 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:01.037652 14405 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 12 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:01.038537 14405 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:00.986961 14529 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:01.039800 11633 raft_consensus.cc:1261] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260809 15:53:00.948532 14407 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:01.041944 14396 raft_consensus.cc:2781] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:53:01.042505 14407 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.042722 14396 raft_consensus.cc:686] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 6 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:01.019855 14164 consensus_queue.cc:1035] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.023s
I20260809 15:53:01.043365 14407 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:01.044173 14396 consensus_queue.cc:237] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:00.983397 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:00.990773 14306 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:53:01.046444 10927 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 8.
I20260809 15:53:01.047703 10927 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:53:00.982748 14517 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.048619 14495 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Advancing to term 10
I20260809 15:53:00.989226 14094 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.051162 14517 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:01.005349 14540 raft_consensus.cc:2726] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:53:01.052327 14540 raft_consensus.cc:2726] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:01.022881 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:01.024596 11640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41482 (request call id 756) took 1983 ms. Trace:
I20260809 15:53:00.948223 14382 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:00.959378 14468 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:53:01.030061 10975 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.791s user 0.007s sys 0.016s
I20260809 15:53:01.039287 14405 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.052666 14094 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:01.036585 10842 rpcz_store.cc:276] 0809 15:52:58.874855 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:58.875043 (+ 188us) service_pool.cc:227] Handling call
0809 15:52:58.903672 (+ 28629us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 15:52:58.980833 (+ 77161us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:58.980874 (+ 41us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 15:52:58.989388 (+ 8514us) log.cc:842] Serialized 52 byte log entry
0809 15:52:59.147349 (+157961us) raft_consensus.cc:1631] Marking committed up to 2
0809 15:52:59.459178 (+311829us) log.cc:842] Serialized 14 byte log entry
0809 15:52:59.512421 (+ 53243us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:01.028296 (+1515875us) spinlock_profiling.cc:243] Waited 58.5 ms on lock 0x7b54001ed054. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:01.028343 (+ 47us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:01.028397 (+ 54us) raft_consensus.cc:1671] finished
0809 15:53:01.035884 (+ 7487us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:01.036059 (+ 175us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":54785,"prepare.queue_time_us":33301,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":36,"raft.queue_time_us":55274,"raft.run_cpu_time_us":242,"raft.run_wall_time_us":17920,"spinlock_wait_cycles":134453120,"thread_start_us":405334,"threads_started":2,"wal-append.queue_time_us":137818,"wal-append.run_cpu_time_us":13783,"wal-append.run_wall_time_us":1511765}
I20260809 15:53:01.054273 11640 rpcz_store.cc:276] 0809 15:52:59.040842 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:59.041567 (+ 725us) service_pool.cc:227] Handling call
0809 15:53:01.024266 (+1982699us) spinlock_profiling.cc:243] Waited 61.2 ms on lock 0x7b5400590bd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:01.024509 (+ 243us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":140828416}
I20260809 15:53:01.023528 10840 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45578 (request call id 2079) took 2302 ms. Trace:
I20260809 15:53:01.058585 10931 raft_consensus.cc:2379] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 7: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:01.065366 10840 rpcz_store.cc:276] 0809 15:52:58.721494 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:58.721655 (+ 161us) service_pool.cc:227] Handling call
0809 15:52:58.739985 (+ 18330us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:52:59.084659 (+344674us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:01.023038 (+1938379us) spinlock_profiling.cc:243] Waited 344 ms on lock 0x7b54001ee954. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:01.023358 (+ 320us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":791094656}
I20260809 15:53:01.049600 14518 leader_election.cc:288] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:53:01.017519 14312 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 1.146s user 0.019s sys 0.018s
I20260809 15:53:01.035254 14188 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:01.046247 14519 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:01.063963 10929 raft_consensus.cc:1226] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260809 15:53:00.986335 14319 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:01.071051 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:01.075033 11009 raft_consensus.cc:1226] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 6. Current term is 7. Ops: []
W20260809 15:53:01.069072 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 138241us
I20260809 15:53:01.084066 14473 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:00.986865 14503 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.088071 14503 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:00.979938 14499 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.089437 14094 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:01.063974 14521 raft_consensus.cc:482] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:01.090458 14521 raft_consensus.cc:504] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.091913 14499 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:00.989081 14498 consensus_queue.cc:237] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.085886 10927 raft_consensus.cc:595] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:01.094022 10927 raft_consensus.cc:603] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 15:53:01.094193 10931 raft_consensus.cc:1261] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 9 index: 5. (index mismatch)
I20260809 15:53:01.066973 14468 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:01.067617 14312 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 1.196s user 0.020s sys 0.019s
I20260809 15:53:01.099126 10978 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:01.102979 11009 raft_consensus.cc:1261] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260809 15:53:01.063781 14529 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:53:01.124334 11017 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50542 (request call id 2451) took 1635 ms (client timeout 1654 ms). Trace:
I20260809 15:53:01.125097 14529 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:01.126960 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:53:01.124943 11017 rpcz_store.cc:269] 0809 15:52:59.488864 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:59.489055 (+ 191us) service_pool.cc:227] Handling call
0809 15:53:01.124114 (+1635059us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b540002e254. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:01.124273 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":555380096}
I20260809 15:53:01.130607 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:01.107554 11635 raft_consensus.cc:2454] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:53:01.109970 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:01.110953 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:01.110833 14319 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:01.112251 14407 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:01.117367 14401 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:01.107400 14094 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:01.118952 11639 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41478 (request call id 1049) took 1742 ms (client timeout 1664 ms). Trace:
I20260809 15:53:01.119578 14382 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:01.104789 14464 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.120138 14405 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:01.122634 11013 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:39978 (request call id 717) took 2198 ms (client timeout 1700 ms). Trace:
I20260809 15:53:01.123669 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:53:01.137017 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 60218us
W20260809 15:53:01.147701 11639 rpcz_store.cc:269] 0809 15:52:59.376589 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:59.376774 (+ 185us) service_pool.cc:227] Handling call
0809 15:53:01.118692 (+1741918us) spinlock_profiling.cc:243] Waited 402 ms on lock 0x7b54004e70d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:01.118842 (+ 150us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":925624576}
W20260809 15:53:01.176234 11013 rpcz_store.cc:269] 0809 15:52:58.924060 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:58.925817 (+ 1757us) service_pool.cc:227] Handling call
0809 15:53:01.122385 (+2196568us) spinlock_profiling.cc:243] Waited 184 ms on lock 0x7b540004c9d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:01.122489 (+ 104us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":422779264}
I20260809 15:53:01.159399 10923 raft_consensus.cc:1226] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 6. Current term is 7. Ops: []
I20260809 15:53:01.117393 14468 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:01.171270 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 204542us
I20260809 15:53:01.176493 11016 raft_consensus.cc:2454] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 3.
I20260809 15:53:01.180404 10978 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:01.126070 14529 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:01.169538 14391 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.169057 14407 raft_consensus.cc:2726] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:01.178350 14464 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.212076 11009 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:53:01.221676 10978 leader_election.cc:302] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
W20260809 15:53:01.181504 11634 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41478 (request call id 1032) took 3455 ms (client timeout 1843 ms). Trace:
W20260809 15:53:01.223405 11634 rpcz_store.cc:269] 0809 15:52:57.723019 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:58.115068 (+392049us) service_pool.cc:227] Handling call
0809 15:53:01.168426 (+3053358us) spinlock_profiling.cc:243] Waited 928 ms on lock 0x7b54004cf754. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:01.173614 (+ 5188us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2134044032}
I20260809 15:53:01.170964 14401 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.226744 14405 raft_consensus.cc:2726] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:53:01.229835 10978 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:01.160851 14473 consensus_queue.cc:1046] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.052s
I20260809 15:53:01.178013 14521 leader_election.cc:288] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:01.225728 14094 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:01.247753 14401 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:01.266600 14401 consensus_queue.cc:1035] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.001s
I20260809 15:53:01.242938 14407 raft_consensus.cc:2726] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 15:53:01.213832 14468 consensus_queue.cc:1035] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.042s
W20260809 15:53:01.255666 14548 log.cc:924] Time spent T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.066s user 0.000s sys 0.000s
I20260809 15:53:01.233044 14549 raft_consensus.cc:2726] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:53:01.231655 14391 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.234695 14464 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:01.297361 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 168952us
I20260809 15:53:01.319350 14188 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.317641 14495 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.277218 14521 raft_consensus.cc:3032] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:53:01.325251 14529 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:01.292971 14549 raft_consensus.cc:482] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:01.258749 14405 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.305163 14564 consensus_queue.cc:1035] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
W20260809 15:53:01.305039 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 103691us
I20260809 15:53:01.314767 14306 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.398663 10930 raft_consensus.cc:1226] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 9. Current term is 10. Ops: []
I20260809 15:53:01.345976 14498 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.739799 14510 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.667142 11636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 978) took 1840 ms. Trace:
I20260809 15:53:01.350361 14407 raft_consensus.cc:482] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.348754 10891 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:01.397289 10923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 2084) took 1920 ms. Trace:
I20260809 15:53:01.655567 10924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 851) took 2460 ms. Trace:
I20260809 15:53:01.701404 14005 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.740949 14510 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.528234 14267 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.742367 10924 rpcz_store.cc:276] 0809 15:52:59.195492 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:59.847329 (+651837us) service_pool.cc:227] Handling call
0809 15:53:01.654568 (+1807239us) spinlock_profiling.cc:243] Waited 7.43 ms on lock 0x7b58000d1498. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f7966593851 00007f7966593702 00007f796f483b09 00007f796e3bf730 00007f796e48579b 00007f796e485821 00007f796e4492b0 00007f796e4347f9 00007f796e4342a2 00007f7967b1a684
0809 15:53:01.654863 (+ 295us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:01.655108 (+ 245us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:01.655147 (+ 39us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:01.655206 (+ 59us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:53:01.655261 (+ 55us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:01.655336 (+ 75us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:01.655490 (+ 154us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":17092864}
I20260809 15:53:01.732636 14520 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 11 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.644820 14283 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:01.744771 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:01.696162 11639 raft_consensus.cc:1261] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260809 15:53:01.745694 10924 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:53:01.746353 14503 raft_consensus.cc:2781] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:53:01.746663 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:01.742763 14005 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.746984 14503 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:01.746891 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:01.747695 14503 raft_consensus.cc:3037] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:53:01.747924 11639 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Advancing to term 10
I20260809 15:53:01.749125 14005 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:53:01.436004 14213 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 5 current term: 5 is higher.
I20260809 15:53:01.439265 11642 raft_consensus.cc:1261] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260809 15:53:01.659653 14247 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:01.432770 14428 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:01.751945 14428 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.350289 14521 raft_consensus.cc:729] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:01.705479 14374 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.754012 14521 consensus_queue.cc:260] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.495332 14571 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:01.755497 14521 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:01.397711 14391 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:01.707142 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:01.702423 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:01.698494 11637 raft_consensus.cc:1261] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
W20260809 15:53:01.757382 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 408690us
I20260809 15:53:01.752678 11642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 981) took 1906 ms. Trace:
I20260809 15:53:01.757377 11640 raft_consensus.cc:2454] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:53:01.741088 11636 rpcz_store.cc:276] 0809 15:52:59.826688 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:00.495615 (+668927us) service_pool.cc:227] Handling call
0809 15:53:00.495958 (+ 343us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:00.555234 (+ 59276us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:00.555257 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:00.555715 (+ 458us) log.cc:842] Serialized 26 byte log entry
0809 15:53:00.816029 (+260314us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:53:00.847439 (+ 31410us) log.cc:842] Serialized 14 byte log entry
0809 15:53:01.666376 (+818937us) spinlock_profiling.cc:243] Waited 2.84 ms on lock 0x7b18003f5620. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f79665424ca 00007f796fc23f99 00007f7966406990 00007f79665c8864 00007f79665c8155 00007f79694eb3d0 00007f79694fd576 00007f796950e0d7 00007f796950e058
0809 15:53:01.666579 (+ 203us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:01.666680 (+ 101us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:01.666738 (+ 58us) raft_consensus.cc:1671] finished
0809 15:53:01.666908 (+ 170us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:01.667066 (+ 158us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3269,"prepare.queue_time_us":31168,"prepare.run_cpu_time_us":3848,"prepare.run_wall_time_us":27271,"raft.queue_time_us":886127,"raft.run_cpu_time_us":341,"raft.run_wall_time_us":342,"spinlock_wait_cycles":6524160,"thread_start_us":201146,"threads_started":2,"wal-append.queue_time_us":320128}
I20260809 15:53:01.754940 14374 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.760660 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:01.781502 11016 raft_consensus.cc:1226] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 6. Current term is 7. Ops: []
I20260809 15:53:01.783097 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:53:01.433159 11632 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41488 (request call id 968) took 1717 ms (client timeout 1529 ms). Trace:
W20260809 15:53:01.784291 11632 rpcz_store.cc:269] 0809 15:52:59.715889 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:59.721719 (+ 5830us) service_pool.cc:227] Handling call
0809 15:53:01.432891 (+1711172us) spinlock_profiling.cc:243] Waited 1.21 s on lock 0x7b54005d2854. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:01.433047 (+ 156us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2791332736}
I20260809 15:53:01.606559 11012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2458) took 1574 ms. Trace:
I20260809 15:53:01.664497 11009 raft_consensus.cc:1261] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 15:53:01.761116 11642 rpcz_store.cc:276] 0809 15:52:59.846539 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:00.559896 (+713357us) service_pool.cc:227] Handling call
0809 15:53:01.438799 (+878903us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:01.750666 (+311867us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:01.752438 (+ 1772us) spinlock_profiling.cc:243] Waited 878 ms on lock 0x7b54005d2850. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:01.752619 (+ 181us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2020326784}
W20260809 15:53:01.552642 10839 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45594 (request call id 2332) took 1751 ms (client timeout 1751 ms). Trace:
W20260809 15:53:01.788897 10839 rpcz_store.cc:269] 0809 15:52:59.801157 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:59.801331 (+ 174us) service_pool.cc:227] Handling call
0809 15:53:01.552400 (+1751069us) spinlock_profiling.cc:243] Waited 241 ms on lock 0x7b54001720d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:01.552550 (+ 150us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":554233600}
I20260809 15:53:01.695186 11634 raft_consensus.cc:1261] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 7. (term mismatch)
W20260809 15:53:01.637562 11641 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41482 (request call id 743) took 3943 ms (client timeout 1935 ms). Trace:
W20260809 15:53:01.567011 10926 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54224 (request call id 2085) took 2090 ms (client timeout 1603 ms). Trace:
W20260809 15:53:01.791219 10926 rpcz_store.cc:269] 0809 15:52:59.477020 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:59.510528 (+ 33508us) service_pool.cc:227] Handling call
0809 15:53:01.554045 (+2043517us) spinlock_profiling.cc:243] Waited 13.1 ms on lock 0x7b540034f154. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:01.566901 (+ 12856us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":30082432}
W20260809 15:53:01.469446 14483 log.cc:924] Time spent T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 1.591s user 0.000s sys 0.002s
I20260809 15:53:01.491288 11635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41488 (request call id 973) took 1773 ms. Trace:
W20260809 15:53:01.793249 11641 rpcz_store.cc:269] 0809 15:52:57.693719 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:58.186592 (+492873us) service_pool.cc:227] Handling call
0809 15:53:01.637322 (+3450730us) spinlock_profiling.cc:243] Waited 2.28 s on lock 0x7b54004e0a54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:01.637472 (+ 150us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5241799936}
I20260809 15:53:01.795680 11635 rpcz_store.cc:276] 0809 15:52:59.717376 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:59.731832 (+ 14456us) service_pool.cc:227] Handling call
0809 15:53:01.491050 (+1759218us) spinlock_profiling.cc:243] Waited 1.26 s on lock 0x7b540066fdd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:01.491196 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2903429888}
W20260809 15:53:01.497900 11011 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50542 (request call id 2450) took 2150 ms (client timeout 2197 ms). Trace:
W20260809 15:53:01.796703 11011 rpcz_store.cc:269] 0809 15:52:59.347548 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:59.347966 (+ 418us) service_pool.cc:227] Handling call
0809 15:53:01.497653 (+2149687us) spinlock_profiling.cc:243] Waited 114 ms on lock 0x7b54000163d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:01.497826 (+ 173us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":263206784}
I20260809 15:53:01.611001 10929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 852) took 2327 ms. Trace:
I20260809 15:53:01.743592 14267 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.797777 10929 rpcz_store.cc:276] 0809 15:52:59.283771 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:59.969099 (+685328us) service_pool.cc:227] Handling call
0809 15:52:59.969399 (+ 300us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:01.069235 (+1099836us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:01.610678 (+541443us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b540021b1d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:01.610930 (+ 252us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2516709248}
I20260809 15:53:01.798967 10930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 2095) took 1290 ms. Trace:
I20260809 15:53:01.593401 10921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54236 (request call id 2102) took 2461 ms. Trace:
I20260809 15:53:01.668704 11606 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:53:01.350766 10847 raft_consensus.cc:1226] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [term 8 LEADER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260809 15:53:01.764848 14213 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.610114 14192 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:53:01.556221 10893 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.84.1:46615 timed out after 1.751s (SENT) [suppressed 27 similar messages]
W20260809 15:53:01.808990 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 396679us
I20260809 15:53:01.766642 14561 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.768850 14345 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:53:01.790064 14164 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:01.771795 14331 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.772140 14300 raft_consensus.cc:482] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:01.750613 14510 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:01.757661 10923 rpcz_store.cc:276] 0809 15:52:59.476733 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:59.994445 (+517712us) service_pool.cc:227] Handling call
0809 15:53:00.024284 (+ 29839us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:01.178783 (+1154499us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:01.396958 (+218175us) spinlock_profiling.cc:243] Waited 1.12 s on lock 0x7b540030a7d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:01.397205 (+ 247us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2569594752}
I20260809 15:53:01.740381 14498 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:01.748899 14306 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:01.688804 14362 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.696970 14537 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.829356 14536 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:53:01.398159 14479 log.cc:924] Time spent T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 1.671s user 0.000s sys 0.008s
I20260809 15:53:01.569989 14396 raft_consensus.cc:979] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
W20260809 15:53:01.548697 14568 log.cc:924] Time spent T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.081s user 0.001s sys 0.000s
W20260809 15:53:01.571969 14459 log.cc:924] Time spent T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 2.185s user 0.005s sys 0.000s
I20260809 15:53:01.794277 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:01.667377 14348 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:01.832841 11636 raft_consensus.cc:2454] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:53:01.645372 14047 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:53:01.785149 11012 rpcz_store.cc:276] 0809 15:53:00.032151 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:00.032336 (+ 185us) service_pool.cc:227] Handling call
0809 15:53:00.032727 (+ 391us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:00.033157 (+ 430us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 15:53:00.165330 (+132173us) log.cc:842] Serialized 14 byte log entry
0809 15:53:00.419431 (+254101us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:00.420031 (+ 600us) log.cc:842] Serialized 26 byte log entry
0809 15:53:00.420191 (+ 160us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:53:00.420234 (+ 43us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:00.420324 (+ 90us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:01.600881 (+1180557us) spinlock_profiling.cc:243] Waited 2.1 ms on lock 0x7b18003b6a40. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f79665cb764 00007f79665ceb11 00007f79665ceac9 00007f79665cea59 00007f79665cea21 00007f79665cdd1d 00007f796f82f204 00007f796f82f039 00007f79665addc6
0809 15:53:01.602777 (+ 1896us) spinlock_profiling.cc:243] Waited 10.4 ms on lock 0x7b18003b6a40. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f79665cb638 00007f79665ceb11 00007f79665ceac9 00007f79665cea59 00007f79665cea21 00007f79665cdd1d 00007f796f82f204 00007f796f82f039 00007f79665addc6
0809 15:53:01.606281 (+ 3504us) raft_consensus.cc:1671] finished
0809 15:53:01.606308 (+ 27us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:01.606483 (+ 175us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":73677,"prepare.queue_time_us":398859,"prepare.run_cpu_time_us":140,"prepare.run_wall_time_us":4839,"raft.queue_time_us":156968,"raft.run_cpu_time_us":279,"raft.run_wall_time_us":17946,"spinlock_wait_cycles":28628992,"thread_start_us":292980,"threads_started":2,"wal-append.queue_time_us":303756}
I20260809 15:53:01.349715 14401 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:01.667496 14455 raft_consensus.cc:3037] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:01.592172 14297 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:01.477777 14388 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.608103 14188 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:01.543936 14530 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.674501 14531 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:01.492493 14280 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:01.362884 14342 consensus_queue.cc:1035] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.439s
I20260809 15:53:01.398290 14495 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:01.744772 14520 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 11 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.787317 14473 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.745746 14283 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:01.363781 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:01.944690 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:01.945158 10920 raft_consensus.cc:3037] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:53:01.835096 14362 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.946812 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:01.947851 10841 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 8.
I20260809 15:53:01.799851 10921 rpcz_store.cc:276] 0809 15:52:59.112337 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:59.118541 (+ 6204us) service_pool.cc:227] Handling call
0809 15:52:59.118957 (+ 416us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:52:59.125767 (+ 6810us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:52:59.125804 (+ 37us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:52:59.126224 (+ 420us) log.cc:842] Serialized 26 byte log entry
0809 15:52:59.228475 (+102251us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:52:59.252624 (+ 24149us) log.cc:842] Serialized 14 byte log entry
0809 15:52:59.252895 (+ 271us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:52:59.253222 (+ 327us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:01.394462 (+2141240us) spinlock_profiling.cc:243] Waited 3.8 ms on lock 0x7b1800109fc0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f79665cb764 00007f79665ceb11 00007f79665ceac9 00007f79665cea59 00007f79665cea21 00007f79665cdd1d 00007f796f82f204 00007f796f82f039 00007f79665addc6
0809 15:53:01.565165 (+170703us) spinlock_profiling.cc:243] Waited 1.93 ms on lock 0x7b1800109fc0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f79665cb638 00007f79665ceb11 00007f79665ceac9 00007f79665cea59 00007f79665cea21 00007f79665cdd1d 00007f796f82f204 00007f796f82f039 00007f79665addc6
0809 15:53:01.573238 (+ 8073us) raft_consensus.cc:1671] finished
0809 15:53:01.573268 (+ 30us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:01.573428 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":19649,"prepare.queue_time_us":17104,"prepare.run_cpu_time_us":49,"prepare.run_wall_time_us":50,"raft.queue_time_us":6885,"raft.run_cpu_time_us":268,"raft.run_wall_time_us":20368,"spinlock_wait_cycles":13174784,"thread_start_us":81694,"threads_started":1,"wal-append.queue_time_us":131628}
I20260809 15:53:01.949077 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:01.950519 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:01.799057 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:01.429621 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:01.823674 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:01.952148 10848 raft_consensus.cc:1261] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 15:53:01.813300 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:01.953013 10931 raft_consensus.cc:2454] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:53:01.953861 11009 raft_consensus.cc:2379] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 9: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:01.821722 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:01.955446 14048 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.955816 11010 raft_consensus.cc:2454] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:53:01.956737 13973 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.956640 14048 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:01.481273 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:01.959944 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:01.961026 10927 raft_consensus.cc:2379] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 5: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:01.816820 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:01.831378 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:01.922672 10840 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:53:01.963393 10840 raft_consensus.cc:595] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:01.792364 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:53:01.807482 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 138874us
I20260809 15:53:01.791923 11634 pending_rounds.cc:76] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9: Aborting all ops after (but not including) 6
I20260809 15:53:01.965143 11634 pending_rounds.cc:97] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9: Aborting uncommitted NO_OP operation due to leader change: 6.7
I20260809 15:53:01.926294 10847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 833) took 1670 ms. Trace:
I20260809 15:53:01.799649 10930 rpcz_store.cc:276] 0809 15:53:00.508134 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:00.952000 (+443866us) service_pool.cc:227] Handling call
0809 15:53:00.952315 (+ 315us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:01.786471 (+834156us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:01.798672 (+ 12201us) spinlock_profiling.cc:243] Waited 446 ms on lock 0x7b540030a2d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:01.798896 (+ 224us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1025943424}
I20260809 15:53:01.965714 11634 raft_consensus.cc:2866] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:53:01.965957 10847 rpcz_store.cc:276] 0809 15:53:00.257124 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:00.257293 (+ 169us) service_pool.cc:227] Handling call
0809 15:53:00.257635 (+ 342us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:01.801242 (+1543607us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:01.925486 (+124244us) spinlock_profiling.cc:243] Waited 1.09 s on lock 0x7b5400171bd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:01.926039 (+ 553us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2513202944}
I20260809 15:53:01.804718 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:01.778066 11639 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 10.
I20260809 15:53:01.812024 14005 raft_consensus.cc:2781] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:53:01.968667 11637 raft_consensus.cc:2362] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6. Re-sending same reply.
I20260809 15:53:01.969354 14005 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:01.970232 14005 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
W20260809 15:53:01.972072 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 415748us
I20260809 15:53:01.839825 14192 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:01.830569 14267 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:01.975008 14192 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.976155 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:01.977043 11637 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:01.813060 14213 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:01.980695 14213 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:01.817349 14345 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:01.984272 14345 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:01.838055 14348 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.986292 10891 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:53:01.987502 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 57915.1us
I20260809 15:53:01.988291 14348 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:01.846779 14388 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.790642 14405 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.786300 14407 raft_consensus.cc:504] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.989308 10978 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:01.991786 14407 leader_election.cc:288] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:01.991925 14388 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:01.992686 14388 raft_consensus.cc:2781] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:53:01.993142 14388 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:01.993625 14388 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:01.838580 14300 raft_consensus.cc:504] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.992698 14405 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:01.994555 11013 raft_consensus.cc:1261] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:53:01.995047 14405 raft_consensus.cc:2781] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:53:01.995617 14405 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:01.995946 11636 raft_consensus.cc:1261] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch)
I20260809 15:53:01.996116 14405 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:01.996321 14300 leader_election.cc:288] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:01.542598 14216 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:01.819799 14164 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.839843 14401 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:01.997323 14216 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:53:01.931315 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:01.998728 10842 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 11 FOLLOWER]: Advancing to term 12
I20260809 15:53:02.000025 14401 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:01.819908 14331 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:01.992448 14407 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:01.945909 14283 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:01.860960 14374 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:01.941105 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:01.950924 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:01.951908 14033 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:01.843279 14297 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.960805 14254 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:01.963560 11015 raft_consensus.cc:2379] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 12: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:02.001394 14407 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.831264 14428 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:01.967336 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:01.771010 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.055764 14033 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.057495 11018 raft_consensus.cc:2396] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 5 because replica has last-logged OpId of term: 4 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 3.
I20260809 15:53:01.993489 10891 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:01.996829 10923 raft_consensus.cc:2379] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 7: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:53:01.826714 14436 consensus_queue.cc:1046] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.006s
W20260809 15:53:01.817708 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 144239us
I20260809 15:53:02.063448 14436 raft_consensus.cc:3032] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:53:02.064168 14436 raft_consensus.cc:729] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:01.818053 14529 consensus_queue.cc:1046] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260809 15:53:01.986068 10737 catalog_manager.cc:5495] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 4 to 6. New cstate: current_term: 6 leader_uuid: "c7aa762318cb4e97875333f10679a9f9" committed_config { opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } } }
I20260809 15:53:02.067253 14529 raft_consensus.cc:3032] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 LEADER]: Stepping down as leader of term 7
I20260809 15:53:02.067816 14529 raft_consensus.cc:729] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
W20260809 15:53:02.068429 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 116817us
I20260809 15:53:01.830192 14536 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:01.829962 14537 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.869669 14520 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 12 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:02.076413 14537 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:02.077173 14537 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:02.069514 14436 consensus_queue.cc:260] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:02.082479 11633 raft_consensus.cc:3037] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:01.854471 14280 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:01.939791 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.086941 14213 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
W20260809 15:53:02.087427 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 94278.9us
I20260809 15:53:02.089227 14280 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:01.739140 14549 raft_consensus.cc:504] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.853510 14530 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.092356 14342 consensus_queue.cc:1035] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:53:02.093276 14530 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:01.816274 14561 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.954011 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:02.121559 10848 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:53:01.993757 11634 raft_consensus.cc:1261] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260809 15:53:02.005229 14331 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:02.022619 14192 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:02.006753 14362 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:02.055330 13806 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:53:02.056691 14503 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.930753 10924 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 11 FOLLOWER]: Advancing to term 12
I20260809 15:53:02.081058 14536 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:02.071216 14482 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:02.081633 14345 consensus_queue.cc:1035] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 15:53:02.159919 14482 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.070611 14487 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:02.161381 14487 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.163772 10809 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:02.163954 14487 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:02.164914 10809 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:02.092613 14549 leader_election.cc:288] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:01.755839 14571 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:01.955353 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:02.202435 14546 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:02.203227 14546 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.204492 14571 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:01.957612 13973 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:02.093657 10838 raft_consensus.cc:2454] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:53:02.082482 14527 raft_consensus.cc:979] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536: : Instructing follower d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:53:02.106491 14254 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.113221 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:02.077772 14537 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:01.862830 14544 raft_consensus.cc:1067] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:53:01.862794 14498 leader_election.cc:288] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:02.133145 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:02.144536 14561 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:02.120908 14436 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:02.069387 14529 consensus_queue.cc:260] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:02.150417 14164 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:02.150996 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:02.151057 13806 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.156313 14407 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:02.157675 14033 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:02.159076 10978 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:02.183656 14297 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:02.156198 14216 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.184092 14261 raft_consensus.cc:2781] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:53:02.184739 14540 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:02.194355 10809 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:02.194674 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.196218 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:53:02.196388 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 55256.9us
I20260809 15:53:02.197810 11009 raft_consensus.cc:1261] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260809 15:53:02.175113 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:02.211027 10842 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 12.
I20260809 15:53:02.202842 10931 raft_consensus.cc:2454] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:53:02.221180 10848 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 9.
I20260809 15:53:02.133648 14348 consensus_queue.cc:1035] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.098s
I20260809 15:53:02.228238 10921 raft_consensus.cc:2354] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5.
I20260809 15:53:02.242640 14048 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:02.243610 11015 raft_consensus.cc:1261] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260809 15:53:02.217392 14482 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:02.244009 11641 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 11 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 11.
I20260809 15:53:01.853065 14531 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.240267 11637 raft_consensus.cc:2454] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 8.
I20260809 15:53:01.859498 14473 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:53:02.267238 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 153225us
I20260809 15:53:02.283754 14261 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:02.255582 14546 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:02.302138 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 107910us
I20260809 15:53:02.297789 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.287781 11013 raft_consensus.cc:2454] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 3.
I20260809 15:53:02.317065 10927 raft_consensus.cc:2379] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 9: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:02.316905 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:53:02.313326 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 154389us
I20260809 15:53:02.317790 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:02.323292 10893 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:02.324456 14561 raft_consensus.cc:2726] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:02.325543 14561 consensus_queue.cc:1035] T 5b472f5780ac4f56a2443e8ccc49125f P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260809 15:53:02.325971 14280 consensus_queue.cc:1035] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 15:53:02.326596 10893 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:53:02.326481 14498 consensus_queue.cc:1035] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.001s
I20260809 15:53:02.327574 14561 raft_consensus.cc:2781] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:53:02.331614 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.331939 10842 raft_consensus.cc:1261] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260809 15:53:02.334738 11011 raft_consensus.cc:2396] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 8 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
I20260809 15:53:02.297670 14261 raft_consensus.cc:3037] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:02.342869 14543 raft_consensus.cc:1067] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Signalling peer d74944fa9fa14ec78b0640ea709ff3aa to start an election
I20260809 15:53:02.344707 11010 raft_consensus.cc:2454] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:53:02.345777 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.346387 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.347004 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.347229 11012 raft_consensus.cc:2454] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:53:02.291755 14540 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:02.348553 11636 raft_consensus.cc:2396] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 9 because replica has last-logged OpId of term: 7 index: 8, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 7.
I20260809 15:53:02.348491 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.343302 14216 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:02.352954 14540 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:02.354527 11017 raft_consensus.cc:1261] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 5. (term mismatch)
I20260809 15:53:02.354959 11017 pending_rounds.cc:76] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa: Aborting all ops after (but not including) 4
I20260809 15:53:02.355247 11017 pending_rounds.cc:97] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa: Aborting uncommitted NO_OP operation due to leader change: 6.5
I20260809 15:53:02.355544 11017 raft_consensus.cc:2866] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:53:02.356179 11017 pending_rounds.cc:97] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa: Aborting uncommitted NO_OP operation due to leader change: 8.6
I20260809 15:53:02.355469 14261 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.356551 11017 raft_consensus.cc:2866] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:53:02.358596 14261 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:02.359078 10893 leader_election.cc:302] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:02.360076 14216 raft_consensus.cc:2726] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:53:02.360415 14388 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:02.336755 14048 raft_consensus.cc:2726] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:53:02.371663 14388 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:02.371778 14473 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:02.372578 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.307180 13973 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:02.379230 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.340094 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.333657 14561 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:02.384289 10931 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:53:02.395646 11013 raft_consensus.cc:2379] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 9: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:02.399497 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:02.401275 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.404378 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:02.413069 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.416993 10842 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 7.
I20260809 15:53:02.396279 14561 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:02.418385 11018 raft_consensus.cc:1261] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260809 15:53:02.420953 11633 raft_consensus.cc:2454] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:53:02.408223 14546 raft_consensus.cc:2781] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:53:02.422874 14537 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:02.422935 14546 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:02.423666 14546 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:02.392213 14216 consensus_queue.cc:1035] T 73ac44823c384ed99817c1ba3bba75ea P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.034s
W20260809 15:53:02.427111 14455 consensus_meta.cc:287] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.585s user 0.003s sys 0.015s
I20260809 15:53:02.398661 10920 raft_consensus.cc:2454] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:53:02.427909 10920 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 2126) took 1165 ms. Trace:
I20260809 15:53:02.428393 14261 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:02.429054 14261 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:02.430226 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:02.430992 10931 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 8.
I20260809 15:53:02.431172 11015 raft_consensus.cc:3037] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:02.431903 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:02.432660 10931 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 9.
I20260809 15:53:02.433671 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:02.434553 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.434695 10893 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:53:02.434823 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.435263 10837 raft_consensus.cc:2454] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:53:02.435695 11013 raft_consensus.cc:2379] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 12: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:02.436206 14473 consensus_queue.cc:1035] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 15:53:02.436607 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:02.437215 10844 raft_consensus.cc:3037] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:02.437397 14503 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:02.438251 14503 raft_consensus.cc:2781] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260809 15:53:02.438689 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.428252 10920 rpcz_store.cc:276] 0809 15:53:01.262372 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:01.340696 (+ 78324us) service_pool.cc:227] Handling call
0809 15:53:02.427788 (+1087092us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:53:02.439973 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.440716 10843 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:53:02.441679 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:02.340339 14487 consensus_queue.cc:1035] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.058s
I20260809 15:53:02.443593 10926 raft_consensus.cc:2379] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:02.443470 14540 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:02.440847 14537 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:02.450412 14537 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.450647 14005 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.452466 14537 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:02.452605 14005 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:02.439949 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:02.453219 10920 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:53:02.453737 14507 consensus_queue.cc:1035] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260809 15:53:02.455266 14531 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:02.393646 14254 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:02.444715 14394 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:02.445011 14405 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:02.445277 13973 consensus_queue.cc:1035] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 15:53:02.436420 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.438789 14503 raft_consensus.cc:686] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 10 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:02.436230 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:02.444316 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.461773 14394 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:02.460623 10924 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 12.
I20260809 15:53:02.470952 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:02.449685 14540 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.475864 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.476722 10920 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 9.
I20260809 15:53:02.479319 10843 raft_consensus.cc:3037] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:53:02.480777 11011 raft_consensus.cc:2396] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 7 because replica has last-logged OpId of term: 6 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 3.
I20260809 15:53:02.482149 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.485412 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.488483 10893 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:53:02.492179 10842 raft_consensus.cc:3037] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:02.473515 14261 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:02.491118 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.478425 14297 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:02.497474 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.497697 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.482188 14473 consensus_queue.cc:1035] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.022s
I20260809 15:53:02.503965 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.504549 11017 raft_consensus.cc:2454] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:53:02.502388 14280 raft_consensus.cc:2781] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 15:53:02.510818 10930 raft_consensus.cc:1226] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 6. Current term is 7. Ops: []
I20260809 15:53:02.512125 10848 raft_consensus.cc:2454] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:53:02.516297 10922 raft_consensus.cc:1261] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 15:53:02.502197 14374 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:02.520937 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:02.496260 14213 raft_consensus.cc:2726] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:02.524922 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:02.525610 11641 raft_consensus.cc:2454] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:53:02.475188 14507 consensus_queue.cc:1046] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.035s
I20260809 15:53:02.530604 10930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 2101) took 1534 ms. Trace:
I20260809 15:53:02.527886 10837 raft_consensus.cc:1261] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260809 15:53:02.511340 14345 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:02.479482 14455 consensus_queue.cc:1035] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.041s
I20260809 15:53:02.531124 10930 rpcz_store.cc:276] 0809 15:53:00.984364 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:02.463374 (+1479010us) service_pool.cc:227] Handling call
0809 15:53:02.510599 (+ 47225us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:02.517254 (+ 6655us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:02.517491 (+ 237us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:53:02.537851 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.538676 10978 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:02.545104 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:02.549719 10931 raft_consensus.cc:1226] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: [6.3-6.3]
I20260809 15:53:02.549574 10843 raft_consensus.cc:2454] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:53:02.551249 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.554486 10839 raft_consensus.cc:2379] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 12: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:02.554986 14540 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:02.530246 14297 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.556975 14297 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:02.552980 14507 consensus_queue.cc:1225] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer 20730d75ad954a82b126148f8152a9f3 log is divergent from this leader: its last log entry 8.9 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
W20260809 15:53:02.560307 14521 consensus_meta.cc:287] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.795s user 0.007s sys 0.013s
I20260809 15:53:02.527410 14374 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.562531 10844 raft_consensus.cc:2454] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:53:02.562911 11604 leader_election.cc:302] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:02.563357 14374 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:53:02.563774 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 65942.5us
I20260809 15:53:02.564632 11604 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:02.565622 14537 raft_consensus.cc:2781] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:53:02.566025 14537 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:02.566403 14537 raft_consensus.cc:3037] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:02.566324 14503 consensus_queue.cc:237] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.569273 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:02.569896 10848 raft_consensus.cc:1226] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260809 15:53:02.570955 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:53:02.572125 14247 consensus_meta.cc:287] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.821s user 0.009s sys 0.010s
I20260809 15:53:02.574115 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:02.574399 14510 raft_consensus.cc:2781] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:02.574909 14510 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:02.575299 14510 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:02.575801 10978 leader_election.cc:302] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:02.575565 14537 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.577139 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.563947 14374 raft_consensus.cc:2781] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election won for term 4
I20260809 15:53:02.578627 14374 raft_consensus.cc:482] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:02.579350 14374 raft_consensus.cc:3037] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:53:02.578419 14297 raft_consensus.cc:2781] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election won for term 4
I20260809 15:53:02.581504 14297 raft_consensus.cc:686] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:02.581630 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.583004 14297 consensus_queue.cc:237] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.535228 14254 consensus_queue.cc:1035] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260809 15:53:02.578439 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:02.588225 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.572197 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:02.578361 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.589602 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.540748 14280 raft_consensus.cc:686] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [term 8 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:02.596009 11015 raft_consensus.cc:1261] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260809 15:53:02.588872 10978 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:02.592078 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:02.598290 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.599000 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54236 (request call id 2128) took 1330 ms. Trace:
I20260809 15:53:02.600294 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.599375 10931 rpcz_store.cc:276] 0809 15:53:01.262933 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:02.482574 (+1219641us) service_pool.cc:227] Handling call
0809 15:53:02.528076 (+ 45502us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:02.592547 (+ 64471us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:02.592819 (+ 272us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:53:02.600869 10839 raft_consensus.cc:2454] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:53:02.602798 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.604764 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.598925 11015 raft_consensus.cc:2454] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:53:02.606786 14405 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:02.608282 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.608939 10930 raft_consensus.cc:2379] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 6: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:02.609005 10839 raft_consensus.cc:2379] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 9: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:02.589501 14048 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:02.611436 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.614661 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.577785 14537 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:02.617892 14048 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.625401 14216 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:02.626276 10843 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:53:02.621717 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.630505 11010 raft_consensus.cc:2454] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:53:02.631551 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:02.632195 11010 raft_consensus.cc:3037] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 6
I20260809 15:53:02.626000 14216 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.634161 14388 consensus_queue.cc:1035] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 15:53:02.636170 14216 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:02.612509 14280 consensus_queue.cc:237] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.626941 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.623190 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.621673 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:02.545686 14561 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.643512 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.643149 10842 raft_consensus.cc:2454] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 8.
I20260809 15:53:02.646212 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.646950 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.647246 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:02.648502 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.647701 11015 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
I20260809 15:53:02.617882 14405 raft_consensus.cc:2781] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 15:53:02.651897 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:53:02.653371 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 87451.3us
I20260809 15:53:02.653563 10839 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:53:02.656590 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.656452 14540 consensus_queue.cc:1035] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:53:02.655833 10930 raft_consensus.cc:2454] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
I20260809 15:53:02.657230 10844 raft_consensus.cc:2454] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:53:02.659510 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.659410 10842 raft_consensus.cc:2454] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:53:02.660938 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.657738 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.653080 14405 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:02.665364 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:02.665165 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:02.665915 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:02.667833 11010 raft_consensus.cc:2454] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:53:02.670953 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:02.676668 13806 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:02.677260 14531 consensus_queue.cc:1035] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 15:53:02.678475 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:02.685055 10921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 2127) took 1422 ms. Trace:
I20260809 15:53:02.685479 10921 rpcz_store.cc:276] 0809 15:53:01.262687 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:01.949687 (+687000us) service_pool.cc:227] Handling call
0809 15:53:02.684840 (+735153us) spinlock_profiling.cc:243] Waited 267 ms on lock 0x7b54000a9bd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:02.684994 (+ 154us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":614699392}
I20260809 15:53:02.687134 10921 raft_consensus.cc:1261] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 6 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260809 15:53:02.688743 10926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 2129) took 1425 ms. Trace:
I20260809 15:53:02.689195 10926 rpcz_store.cc:276] 0809 15:53:01.343683 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:01.792149 (+448466us) service_pool.cc:227] Handling call
0809 15:53:02.688479 (+896330us) spinlock_profiling.cc:243] Waited 410 ms on lock 0x7b540034fb54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:02.688656 (+ 177us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":942902016}
I20260809 15:53:02.696203 11604 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:02.697564 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:02.703109 10930 raft_consensus.cc:2379] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 9: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:02.706413 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.707311 11015 raft_consensus.cc:2454] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:53:02.664126 14537 raft_consensus.cc:482] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:02.694577 11635 raft_consensus.cc:3037] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:53:02.667690 14405 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Advancing to term 10
I20260809 15:53:02.717025 10978 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
W20260809 15:53:02.701967 14588 log.cc:924] Time spent T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.139s user 0.000s sys 0.001s
W20260809 15:53:02.727670 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 64780.2us
I20260809 15:53:02.736014 10809 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:02.722316 14005 raft_consensus.cc:2781] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:53:02.748711 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:02.750392 11015 raft_consensus.cc:3037] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:02.749750 10809 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:02.752791 10930 raft_consensus.cc:1261] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 9 index: 9. (index mismatch)
I20260809 15:53:02.753005 13806 consensus_queue.cc:1035] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
I20260809 15:53:02.760907 14345 raft_consensus.cc:2726] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:02.761485 11632 raft_consensus.cc:1261] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 7 index: 9. (index mismatch)
I20260809 15:53:02.761938 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
W20260809 15:53:02.762322 14590 log.cc:924] Time spent T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.105s user 0.001s sys 0.001s
I20260809 15:53:02.762863 11640 raft_consensus.cc:2379] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 6: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:02.763062 10930 raft_consensus.cc:1261] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 3. Preceding OpId from leader: term: 6 index: 4. (index mismatch)
I20260809 15:53:02.753055 14527 raft_consensus.cc:2781] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:53:02.763963 14527 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:02.766614 10809 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:02.767969 14527 raft_consensus.cc:3037] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:53:02.768247 10929 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:53:02.768704 10809 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:02.769627 10809 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:02.770303 14402 raft_consensus.cc:2781] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:53:02.770864 14402 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:02.743726 14407 raft_consensus.cc:482] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:02.771296 14402 raft_consensus.cc:3037] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:02.771517 14407 raft_consensus.cc:504] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.772677 14394 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:02.773526 14407 leader_election.cc:288] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:02.774293 14164 raft_consensus.cc:2781] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:02.774744 14164 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:02.775094 14164 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:02.779021 10922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54236 (request call id 2124) took 2456 ms. Trace:
I20260809 15:53:02.779408 10922 rpcz_store.cc:276] 0809 15:53:00.322330 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:00.390349 (+ 68019us) service_pool.cc:227] Handling call
0809 15:53:00.390696 (+ 347us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:02.518506 (+2127810us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:02.778684 (+260178us) spinlock_profiling.cc:243] Waited 2.1 s on lock 0x7b54000a96d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:02.778943 (+ 259us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4827224832}
I20260809 15:53:02.770221 11642 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8"
dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34950
I20260809 15:53:02.781463 11642 raft_consensus.cc:595] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:02.781898 11642 raft_consensus.cc:603] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 15:53:02.747102 14531 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:02.790778 14531 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.791344 10922 raft_consensus.cc:1261] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 7 index: 5. (index mismatch)
I20260809 15:53:02.792603 10922 raft_consensus.cc:1226] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260809 15:53:02.792904 14531 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:02.793540 14123 raft_consensus.cc:2781] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:53:02.793792 10922 raft_consensus.cc:1226] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 7. Current term is 8. Ops: [7.8-7.8]
I20260809 15:53:02.793936 14123 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:02.727507 14546 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:02.752343 14005 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:02.740422 14537 raft_consensus.cc:504] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.798874 14394 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:02.728650 14254 consensus_queue.cc:1035] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260809 15:53:02.800248 11635 raft_consensus.cc:2454] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:53:02.801447 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:02.803526 14123 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:02.799567 13806 raft_consensus.cc:2781] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:53:02.816951 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:02.820803 11635 raft_consensus.cc:2379] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 7: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:53:02.814744 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:02.813668 14005 raft_consensus.cc:3037] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:02.761958 14510 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:02.824546 11640 raft_consensus.cc:3037] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:53:02.823659 11632 raft_consensus.cc:2354] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5.
I20260809 15:53:02.815516 14561 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:02.835552 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:02.852788 14033 raft_consensus.cc:3032] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 9 LEADER]: Stepping down as leader of term 9
I20260809 15:53:02.852945 14546 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:02.853247 14033 raft_consensus.cc:729] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:02.854669 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:02.825794 13806 raft_consensus.cc:686] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:02.860203 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:02.860235 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.863687 11018 raft_consensus.cc:2379] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 7: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:02.836460 14394 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.875173 14048 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:02.876972 11014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2261) took 1058 ms. Trace:
I20260809 15:53:02.879833 11014 rpcz_store.cc:276] 0809 15:53:01.818817 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:01.818947 (+ 130us) service_pool.cc:227] Handling call
0809 15:53:01.829922 (+ 10975us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:02.002699 (+172777us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:02.002784 (+ 85us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:02.036122 (+ 33338us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:53:02.036213 (+ 91us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:02.876622 (+840409us) spinlock_profiling.cc:243] Waited 172 ms on lock 0x7b54000127d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:02.876666 (+ 44us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:02.876787 (+ 121us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":394948352}
I20260809 15:53:02.881528 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.885022 14529 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:02.885864 10811 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:02.886914 14048 raft_consensus.cc:2726] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260809 15:53:02.887349 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:02.893383 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.894733 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:02.895043 11016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2255) took 1111 ms. Trace:
I20260809 15:53:02.895385 11014 raft_consensus.cc:3037] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 5
I20260809 15:53:02.895536 11016 rpcz_store.cc:276] 0809 15:53:01.783355 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:01.783707 (+ 352us) service_pool.cc:227] Handling call
0809 15:53:01.784006 (+ 299us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:02.203349 (+419343us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:02.203401 (+ 52us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:02.203452 (+ 51us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:53:02.203488 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:02.894654 (+691166us) spinlock_profiling.cc:243] Waited 419 ms on lock 0x7b540004d3d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:02.894709 (+ 55us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:02.894858 (+ 149us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":963467520}
I20260809 15:53:02.897235 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:02.865749 14561 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:53:02.898664 14561 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:02.900795 10927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54224 (request call id 2104) took 1127 ms. Trace:
I20260809 15:53:02.900921 10811 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:53:02.877023 14394 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:02.853600 14374 raft_consensus.cc:504] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.901470 10927 rpcz_store.cc:276] 0809 15:53:01.773122 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:02.096185 (+323063us) service_pool.cc:227] Handling call
0809 15:53:02.900569 (+804384us) spinlock_profiling.cc:243] Waited 72 ms on lock 0x7b540025d0d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:02.900715 (+ 146us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":165674368}
I20260809 15:53:02.900723 14561 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:02.913951 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:02.917369 11640 raft_consensus.cc:2454] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:53:02.920310 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.907581 14048 raft_consensus.cc:2781] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:53:02.923617 10811 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:02.929265 14492 raft_consensus.cc:2726] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:02.929688 14247 consensus_queue.cc:1046] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 6.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:53:02.930428 10811 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:02.930903 14492 raft_consensus.cc:3032] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:53:02.927912 11635 raft_consensus.cc:2379] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 4: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:53:02.932976 10811 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:02.926662 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.849887 11015 raft_consensus.cc:2454] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 7.
I20260809 15:53:02.936340 11604 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:02.863004 14033 consensus_queue.cc:260] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.962869 11642 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:53:02.964073 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.979893 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:53:02.980191 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 394991us
I20260809 15:53:02.971105 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:02.931360 14492 raft_consensus.cc:729] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:02.982774 11015 raft_consensus.cc:2454] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
W20260809 15:53:02.985293 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 47033.6us
I20260809 15:53:02.906487 14402 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:02.933881 14048 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:02.931514 14247 raft_consensus.cc:2726] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:02.992573 10891 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:02.993944 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:02.982964 14348 raft_consensus.cc:2781] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:53:02.996850 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.000135 11642 raft_consensus.cc:2454] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:53:03.004925 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:03.007449 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:03.013797 11016 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:53:03.008741 14394 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:03.015584 14564 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:03.015705 14394 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:03.016340 14564 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:03.017979 14394 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:03.019141 13973 consensus_queue.cc:1035] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:53:03.022317 14394 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:03.022603 10893 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:53:03.022894 14394 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:03.025117 14394 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:03.025385 10893 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:53:03.026230 14394 raft_consensus.cc:2781] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:53:03.026564 14521 raft_consensus.cc:2781] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:53:03.027127 10893 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:03.027323 14394 raft_consensus.cc:686] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:03.028656 14394 consensus_queue.cc:237] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:03.030586 10893 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:03.033705 14216 raft_consensus.cc:2726] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:53:03.042501 14492 consensus_queue.cc:260] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:03.044248 10893 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:03.044668 14537 leader_election.cc:288] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:02.991161 14546 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:03.050515 14546 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:03.053020 14546 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:03.056135 14388 raft_consensus.cc:2726] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:53:03.056948 14388 consensus_queue.cc:1035] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 15:53:03.059669 10893 leader_election.cc:302] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:02.993582 14048 raft_consensus.cc:3037] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:02.996546 14033 raft_consensus.cc:3037] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Advancing to term 10
I20260809 15:53:03.076141 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:03.075287 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:03.083024 10893 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 12 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:03.083364 11642 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 11 FOLLOWER]: Advancing to term 12
I20260809 15:53:03.084697 11014 raft_consensus.cc:2454] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:53:03.026194 14463 raft_consensus.cc:2781] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:03.083762 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 2102) took 1304 ms. Trace:
I20260809 15:53:03.085692 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:03.086645 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:03.001924 14510 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:03.012375 14348 raft_consensus.cc:686] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [term 7 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:53:03.085554 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:03.027184 14521 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:03.091853 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:03.044250 14492 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
W20260809 15:53:03.089044 11011 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.504s user 0.008s sys 0.018s
I20260809 15:53:03.090045 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:53:03.104236 11011 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.519s user 0.008s sys 0.018s
I20260809 15:53:03.105422 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 10 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:03.100191 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:03.057540 14405 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:03.049710 14005 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:03.078203 14455 raft_consensus.cc:2726] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:03.086294 14388 raft_consensus.cc:2726] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:03.107581 10840 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:03.106675 11018 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 9.
I20260809 15:53:03.101384 11016 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50528
I20260809 15:53:03.125440 11016 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:53:03.126112 11016 raft_consensus.cc:3037] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:53:03.121762 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:03.130561 11015 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:53:03.132623 11015 raft_consensus.cc:1226] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 5. Current term is 6. Ops: []
I20260809 15:53:03.134931 14261 consensus_queue.cc:1046] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:53:03.136842 10891 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:03.103843 14521 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:03.093459 14463 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:02.993096 13806 consensus_queue.cc:237] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:03.166813 11632 raft_consensus.cc:2354] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 5. Current term is 6.
I20260809 15:53:03.169687 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:03.141038 11633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41488 (request call id 989) took 1415 ms. Trace:
I20260809 15:53:03.093622 10931 rpcz_store.cc:276] 0809 15:53:01.772430 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:02.679653 (+907223us) service_pool.cc:227] Handling call
0809 15:53:02.680041 (+ 388us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:02.680344 (+ 303us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 15:53:02.680790 (+ 446us) log.cc:842] Serialized 14 byte log entry
0809 15:53:03.075555 (+394765us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:03.076059 (+ 504us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:53:03.076125 (+ 66us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:03.076209 (+ 84us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:03.076352 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":3360,"prepare.queue_time_us":855,"prepare.run_cpu_time_us":34,"prepare.run_wall_time_us":36,"raft.queue_time_us":7118,"raft.run_cpu_time_us":143,"raft.run_wall_time_us":7229,"thread_start_us":390763,"threads_started":1}
I20260809 15:53:03.173696 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:03.173808 11633 rpcz_store.cc:276] 0809 15:53:01.703665 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:01.706763 (+ 3098us) service_pool.cc:227] Handling call
0809 15:53:03.092126 (+1385363us) spinlock_profiling.cc:243] Waited 325 ms on lock 0x7b5400134a54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:03.118871 (+ 26745us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":747643776}
I20260809 15:53:03.177711 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:03.178714 11018 raft_consensus.cc:2396] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 8 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 4 index: 6.
I20260809 15:53:03.122735 14216 raft_consensus.cc:2726] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 12 FOLLOWER]: Leader pre-election lost for term 12. Reason: could not achieve majority
I20260809 15:53:03.124571 14564 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:03.185220 14402 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:03.169081 14463 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:03.190229 14402 raft_consensus.cc:3032] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:53:03.195143 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:03.172066 14374 leader_election.cc:288] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:03.197419 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.200135 11633 raft_consensus.cc:2454] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:53:03.201066 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:03.204550 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:03.205307 11014 raft_consensus.cc:3037] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:53:03.206382 10931 raft_consensus.cc:2454] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:53:03.213142 10922 raft_consensus.cc:1226] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 8. Current term is 9. Ops: []
I20260809 15:53:03.194166 14402 raft_consensus.cc:729] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:03.218680 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.233342 10931 raft_consensus.cc:1261] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
W20260809 15:53:03.240338 10926 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.547s user 0.011s sys 0.026s
W20260809 15:53:03.241971 10926 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.549s user 0.011s sys 0.026s
W20260809 15:53:03.270253 10921 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.511s user 0.007s sys 0.013s
W20260809 15:53:03.271919 10921 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.512s user 0.007s sys 0.013s
I20260809 15:53:03.259727 11016 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:03.274150 14473 raft_consensus.cc:2781] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 15:53:03.284487 10920 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54236 (request call id 2134) took 1939 ms. Trace:
I20260809 15:53:03.284976 10920 rpcz_store.cc:276] 0809 15:53:01.345266 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:02.482816 (+1137550us) service_pool.cc:227] Handling call
0809 15:53:02.495627 (+ 12811us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:02.538869 (+ 43242us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:02.538898 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:02.544037 (+ 5139us) log.cc:842] Serialized 26 byte log entry
0809 15:53:02.919111 (+375074us) raft_consensus.cc:1631] Marking committed up to 3
0809 15:53:02.919162 (+ 51us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:02.919245 (+ 83us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:03.284205 (+364960us) raft_consensus.cc:1671] finished
0809 15:53:03.284231 (+ 26us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:03.284397 (+ 166us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1099,"raft.queue_time_us":7686,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":2622,"thread_start_us":346164,"threads_started":1,"wal-append.queue_time_us":658384}
I20260809 15:53:03.190594 14348 consensus_queue.cc:237] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:03.290719 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.263023 14561 raft_consensus.cc:2781] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:53:03.297184 11633 raft_consensus.cc:2454] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:53:03.278806 11016 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:03.296622 11009 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.510s user 0.004s sys 0.020s
W20260809 15:53:03.302983 11009 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.522s user 0.004s sys 0.021s
W20260809 15:53:03.302779 10978 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.552s user 0.019s sys 0.006s
W20260809 15:53:03.353111 10978 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.602s user 0.019s sys 0.006s
W20260809 15:53:03.294631 14297 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.527s user 0.005s sys 0.014s
I20260809 15:53:03.353950 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.380824 11633 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 8.
I20260809 15:53:03.383876 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
W20260809 15:53:03.368294 14297 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.615s user 0.009s sys 0.014s
W20260809 15:53:03.387779 14601 log.cc:924] Time spent T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.093s user 0.000s sys 0.000s
W20260809 15:53:03.388949 14164 consensus_meta.cc:287] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.613s user 0.003s sys 0.006s
I20260809 15:53:03.388870 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.347078 14123 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:03.391058 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.392015 14123 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:03.392297 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.393350 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.394483 11009 raft_consensus.cc:2379] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 6: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:53:03.396353 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.397503 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.398506 11009 raft_consensus.cc:1226] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 6. Current term is 7. Ops: []
I20260809 15:53:03.404711 14521 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:03.394483 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.407655 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.408255 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:53:03.409758 14579 log.cc:924] Time spent T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 1.057s user 0.001s sys 0.001s
W20260809 15:53:03.410692 10924 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 2131) took 2066 ms (client timeout 1872 ms). Trace:
W20260809 15:53:03.411113 10924 rpcz_store.cc:269] 0809 15:53:01.344437 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:01.746444 (+402007us) service_pool.cc:227] Handling call
0809 15:53:03.410450 (+1664006us) spinlock_profiling.cc:243] Waited 183 ms on lock 0x7b540030b6d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:03.410611 (+ 161us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":420742016}
I20260809 15:53:03.413573 10924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:54216 (request call id 1596) took 2358 ms. Trace:
I20260809 15:53:03.351995 14005 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:03.414245 14537 consensus_queue.cc:1046] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
W20260809 15:53:03.414654 10923 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54224 (request call id 2108) took 1600 ms (client timeout 1603 ms). Trace:
I20260809 15:53:03.415350 14537 raft_consensus.cc:3032] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Stepping down as leader of term 4
W20260809 15:53:03.415186 14591 log.cc:924] Time spent T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.857s user 0.000s sys 0.002s
I20260809 15:53:03.415742 14537 raft_consensus.cc:729] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:53:03.416898 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 1 index: 3 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.416694 14537 consensus_queue.cc:260] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:03.417943 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.418658 11632 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 8.
I20260809 15:53:03.419503 14521 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:03.419895 10930 raft_consensus.cc:1261] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260809 15:53:03.409914 11606 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:53:03.420941 14254 raft_consensus.cc:2781] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 15:53:03.421363 14254 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:03.421782 14254 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Advancing to term 10
I20260809 15:53:03.417881 14537 raft_consensus.cc:3037] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:53:03.380327 14402 consensus_queue.cc:260] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:53:03.426793 14605 log.cc:924] Time spent T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.057s user 0.000s sys 0.000s
I20260809 15:53:03.427702 14405 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 10 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:03.431137 14527 consensus_meta.cc:287] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.663s user 0.010s sys 0.001s
I20260809 15:53:03.431217 14254 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:03.431607 14527 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:03.433199 14254 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 10 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:03.433570 14527 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:03.435767 10847 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2334) took 1033 ms. Trace:
I20260809 15:53:03.436627 14402 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
W20260809 15:53:03.378976 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 662002us
I20260809 15:53:03.436236 10847 rpcz_store.cc:276] 0809 15:53:02.401946 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:02.402178 (+ 232us) service_pool.cc:227] Handling call
0809 15:53:02.417180 (+ 15002us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:02.625619 (+208439us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:02.625657 (+ 38us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:02.625695 (+ 38us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:53:02.625736 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:03.435490 (+809754us) spinlock_profiling.cc:243] Waited 208 ms on lock 0x7b5400059bd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:03.435550 (+ 60us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:03.435697 (+ 147us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":463,"raft.run_cpu_time_us":74,"raft.run_wall_time_us":77,"spinlock_wait_cycles":478573440}
I20260809 15:53:03.409651 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.389447 14164 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:03.413911 10924 rpcz_store.cc:276] 0809 15:53:01.055626 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:03.411895 (+2356269us) service_pool.cc:227] Handling call
0809 15:53:03.413500 (+ 1605us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:53:03.442628 10838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 843) took 1003 ms. Trace:
I20260809 15:53:03.419498 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
W20260809 15:53:03.446007 10929 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54236 (request call id 2130) took 2102 ms (client timeout 2099 ms). Trace:
I20260809 15:53:03.427731 11606 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:03.443050 11642 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 12.
W20260809 15:53:03.415081 10923 rpcz_store.cc:269] 0809 15:53:01.814379 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:01.819733 (+ 5354us) service_pool.cc:227] Handling call
0809 15:53:03.414410 (+1594677us) spinlock_profiling.cc:243] Waited 43.6 ms on lock 0x7b540034f154. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:03.414582 (+ 172us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":100327168}
I20260809 15:53:03.444700 10930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 858) took 1434 ms. Trace:
I20260809 15:53:03.448326 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.446995 10978 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:03.448963 10840 raft_consensus.cc:2454] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
W20260809 15:53:03.446445 10929 rpcz_store.cc:269] 0809 15:53:01.344105 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:01.798827 (+454722us) service_pool.cc:227] Handling call
0809 15:53:03.439428 (+1640601us) spinlock_profiling.cc:243] Waited 638 ms on lock 0x7b54003ee254. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:03.445937 (+ 6509us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1467019008}
I20260809 15:53:03.449905 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:03.450506 11633 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:03.447841 10930 rpcz_store.cc:276] 0809 15:53:02.010008 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:02.763844 (+753836us) service_pool.cc:227] Handling call
0809 15:53:03.419582 (+655738us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:03.420376 (+ 794us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:03.444522 (+ 24146us) spinlock_profiling.cc:243] Waited 655 ms on lock 0x7b540034f150. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:03.444647 (+ 125us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1507357440}
I20260809 15:53:03.455394 11632 raft_consensus.cc:2354] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for earlier term 7. Current term is 8.
I20260809 15:53:03.455065 11606 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:03.408322 11635 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
I20260809 15:53:03.443445 11014 raft_consensus.cc:2454] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:53:03.467100 11014 raft_consensus.cc:1226] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260809 15:53:03.467823 14473 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:03.468073 14005 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:53:03.468324 14473 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Advancing to term 10
I20260809 15:53:03.472520 10844 raft_consensus.cc:2354] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8.
I20260809 15:53:03.466020 11604 leader_election.cc:302] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:03.476544 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.443938 14561 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:03.479022 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:03.479548 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.468849 10737 catalog_manager.cc:5495] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 3 to 5, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 5 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } }
I20260809 15:53:03.444372 10838 rpcz_store.cc:276] 0809 15:53:02.439248 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:02.520055 (+ 80807us) service_pool.cc:227] Handling call
0809 15:53:02.534204 (+ 14149us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 15:53:02.553688 (+ 19484us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:02.553734 (+ 46us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 15:53:02.605188 (+ 51454us) log.cc:842] Serialized 52 byte log entry
0809 15:53:02.988002 (+382814us) raft_consensus.cc:1631] Marking committed up to 8
0809 15:53:02.998201 (+ 10199us) log.cc:842] Serialized 14 byte log entry
0809 15:53:02.998392 (+ 191us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:02.998466 (+ 74us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:03.404389 (+405923us) spinlock_profiling.cc:243] Waited 8.56 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:53:03.424459 (+ 20070us) spinlock_profiling.cc:243] Waited 10.8 ms on lock 0x7b1800084fa0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f79665cb638 00007f79665ceb11 00007f79665ceac9 00007f79665cea59 00007f79665cea21 00007f79665cdd1d 00007f796f82f204 00007f796f82f039 00007f79665addc6
0809 15:53:03.442438 (+ 17979us) raft_consensus.cc:1671] finished
0809 15:53:03.442460 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:03.442576 (+ 116us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":32440,"prepare.queue_time_us":10131,"prepare.run_cpu_time_us":45,"prepare.run_wall_time_us":46,"raft.queue_time_us":40912,"raft.run_cpu_time_us":236,"raft.run_wall_time_us":6389,"spinlock_wait_cycles":44629120,"thread_start_us":363053,"threads_started":1,"wal-append.queue_time_us":678661}
I20260809 15:53:03.496249 11636 raft_consensus.cc:2379] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 6: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:03.497043 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:03.473856 14297 raft_consensus.cc:2741] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won
I20260809 15:53:03.500907 11632 raft_consensus.cc:2379] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 6: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:53:03.501379 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.505574 11014 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "0f891fe62df749108fa553cb0661a874"
dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:50542
I20260809 15:53:03.506675 11014 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 15:53:03.507210 11014 raft_consensus.cc:3037] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:53:03.491115 14561 raft_consensus.cc:3037] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
W20260809 15:53:03.505937 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 485945us
W20260809 15:53:03.505249 14503 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.936s user 0.013s sys 0.018s
W20260809 15:53:03.517666 14503 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.948s user 0.013s sys 0.018s
I20260809 15:53:03.520159 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:03.521121 11640 raft_consensus.cc:2379] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 7: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
W20260809 15:53:03.472658 14529 consensus_meta.cc:287] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.587s user 0.010s sys 0.002s
I20260809 15:53:03.528959 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.535046 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.536051 11633 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:53:03.515802 10927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 864) took 1080 ms. Trace:
W20260809 15:53:03.528209 11010 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.760s user 0.010s sys 0.033s
W20260809 15:53:03.541867 11010 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.774s user 0.010s sys 0.033s
I20260809 15:53:03.542754 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.544322 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.544895 11636 raft_consensus.cc:2379] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 12 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 12: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:03.510673 14374 raft_consensus.cc:2781] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election won for term 4
W20260809 15:53:03.553325 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 416520us
I20260809 15:53:03.536788 10927 rpcz_store.cc:276] 0809 15:53:02.435278 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:02.928215 (+492937us) service_pool.cc:227] Handling call
0809 15:53:02.983245 (+ 55030us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:02.983506 (+ 261us) raft_consensus.cc:1520] Early marking committed up to index 4
0809 15:53:02.996421 (+ 12915us) log.cc:842] Serialized 14 byte log entry
0809 15:53:03.515460 (+519039us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:03.515523 (+ 63us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:53:03.515579 (+ 56us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:03.515652 (+ 73us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:03.515746 (+ 94us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":14898,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":976,"thread_start_us":427446,"threads_started":1}
I20260809 15:53:03.555523 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.890941 10891 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:53:03.570546 11013 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.671s user 0.009s sys 0.033s
W20260809 15:53:03.891601 11013 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.992s user 0.015s sys 0.067s
W20260809 15:53:03.554946 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 418158us
W20260809 15:53:03.554715 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 471731us
I20260809 15:53:03.893186 10893 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:03.893703 11640 raft_consensus.cc:2379] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 6: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:53:03.897651 11640 raft_consensus.cc:1226] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 9. Current term is 10. Ops: []
I20260809 15:53:03.818616 14164 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:03.900269 11640 raft_consensus.cc:1261] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 7. (index mismatch)
I20260809 15:53:03.903059 11640 raft_consensus.cc:1261] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 8. (term mismatch)
I20260809 15:53:03.905129 11640 pending_rounds.cc:76] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9: Aborting all ops after (but not including) 7
I20260809 15:53:03.592123 11017 raft_consensus.cc:1261] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260809 15:53:03.906121 11017 pending_rounds.cc:76] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa: Aborting all ops after (but not including) 4
I20260809 15:53:03.615926 10922 raft_consensus.cc:1203] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 10 LEADER]: Deduplicated request from leader. Original: 4.7->[7.8-9.9] Dedup: 7.8->[9.9-9.9]
I20260809 15:53:03.906939 11011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2267) took 1326 ms. Trace:
I20260809 15:53:03.906925 10922 raft_consensus.cc:1226] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [term 10 LEADER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 9. Current term is 10. Ops: [7.8-9.9]
I20260809 15:53:03.907598 11011 rpcz_store.cc:276] 0809 15:53:02.580942 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:02.584071 (+ 3129us) service_pool.cc:227] Handling call
0809 15:53:02.584368 (+ 297us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:02.584600 (+ 232us) raft_consensus.cc:1520] Early marking committed up to index 4
0809 15:53:03.108383 (+523783us) log.cc:842] Serialized 14 byte log entry
0809 15:53:03.906620 (+798237us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:03.906676 (+ 56us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:53:03.906724 (+ 48us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:03.906785 (+ 61us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:03.906883 (+ 98us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":290444,"prepare.queue_time_us":344740,"prepare.run_cpu_time_us":27,"prepare.run_wall_time_us":28,"thread_start_us":966434,"threads_started":2}
I20260809 15:53:03.909592 10922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 2126) took 1036 ms. Trace:
W20260809 15:53:03.651042 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 110006us
W20260809 15:53:03.788251 14607 log.cc:924] Time spent T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.082s user 0.000s sys 0.000s
I20260809 15:53:03.914294 11017 pending_rounds.cc:97] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa: Aborting uncommitted NO_OP operation due to leader change: 7.5
W20260809 15:53:03.592378 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 52167.8us
I20260809 15:53:03.916988 11640 pending_rounds.cc:97] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9: Aborting uncommitted NO_OP operation due to leader change: 7.8
I20260809 15:53:03.917801 11640 raft_consensus.cc:2866] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:53:03.639634 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:03.919912 11640 raft_consensus.cc:1226] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 9. Current term is 10. Ops: []
I20260809 15:53:03.768553 14527 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.176s
I20260809 15:53:03.719201 11014 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:03.539387 10737 catalog_manager.cc:5495] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 1 to 3. New cstate: current_term: 3 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:53:03.616796 14374 raft_consensus.cc:686] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:53:03.527980 14254 raft_consensus.cc:2726] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
W20260809 15:53:03.951408 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 60559.6us
W20260809 15:53:03.840150 10931 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.559s user 0.003s sys 0.026s
W20260809 15:53:03.955938 10931 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.674s user 0.005s sys 0.028s
W20260809 15:53:03.956369 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 63735.9us
W20260809 15:53:03.963399 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 70303.5us
W20260809 15:53:03.953758 11015 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.652s user 0.021s sys 0.027s
W20260809 15:53:03.965812 11015 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.664s user 0.022s sys 0.027s
W20260809 15:53:03.889379 14033 consensus_meta.cc:287] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.797s user 0.007s sys 0.004s
W20260809 15:53:04.001715 11009 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.583s user 0.017s sys 0.045s
W20260809 15:53:04.008715 11009 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.590s user 0.017s sys 0.045s
I20260809 15:53:03.980556 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 2118) took 1472 ms. Trace:
I20260809 15:53:04.028461 10931 rpcz_store.cc:276] 0809 15:53:02.507738 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:03.280344 (+772606us) service_pool.cc:227] Handling call
0809 15:53:03.280680 (+ 336us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:03.280892 (+ 212us) raft_consensus.cc:1520] Early marking committed up to index 6
0809 15:53:03.281231 (+ 339us) log.cc:842] Serialized 14 byte log entry
0809 15:53:03.967418 (+686187us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:03.967497 (+ 79us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:53:03.980243 (+ 12746us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:03.980335 (+ 92us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:03.980486 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":11012,"prepare.queue_time_us":697,"prepare.run_cpu_time_us":60,"prepare.run_wall_time_us":62,"raft.queue_time_us":14394,"raft.run_cpu_time_us":169,"raft.run_wall_time_us":12174,"thread_start_us":674722,"threads_started":1}
W20260809 15:53:03.969832 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 57583.6us
I20260809 15:53:03.981523 10893 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:53:03.914713 11017 raft_consensus.cc:2866] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:53:03.749074 14507 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:53:04.006274 11016 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.571s user 0.006s sys 0.005s
I20260809 15:53:03.916205 10978 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:53:04.003958 11604 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.507s user 0.030s sys 0.050s
W20260809 15:53:04.038981 11016 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.603s user 0.006s sys 0.005s
W20260809 15:53:04.039170 11604 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.544s user 0.030s sys 0.050s
W20260809 15:53:04.039480 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 573585us
W20260809 15:53:04.026954 11606 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.516s user 0.000s sys 0.010s
W20260809 15:53:04.044526 11606 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.534s user 0.000s sys 0.011s
I20260809 15:53:03.937582 10737 catalog_manager.cc:5495] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 3 to 6, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 6 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
W20260809 15:53:04.049063 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 81194.6us
I20260809 15:53:04.049896 10893 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:53:03.966578 10920 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.580s user 0.000s sys 0.010s
W20260809 15:53:04.053490 10920 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.667s user 0.000s sys 0.011s
I20260809 15:53:03.910363 10922 rpcz_store.cc:276] 0809 15:53:02.873433 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:03.307255 (+433822us) service_pool.cc:227] Handling call
0809 15:53:03.381606 (+ 74351us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 15:53:03.908093 (+526487us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:03.909113 (+ 1020us) spinlock_profiling.cc:243] Waited 156 ms on lock 0x7b540030a2d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:03.909461 (+ 348us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":357693824}
I20260809 15:53:04.057181 11016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2482) took 1165 ms. Trace:
W20260809 15:53:04.059221 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 554565us
W20260809 15:53:03.992852 14609 log.cc:924] Time spent T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.341s user 0.000s sys 0.001s
I20260809 15:53:04.066232 11014 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:03.966856 14455 raft_consensus.cc:2781] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
W20260809 15:53:04.004623 14048 consensus_meta.cc:287] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.922s user 0.010s sys 0.001s
W20260809 15:53:04.071431 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 155483us
I20260809 15:53:04.073316 11017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2264) took 1505 ms. Trace:
W20260809 15:53:04.006031 14436 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.567s user 0.002s sys 0.008s
I20260809 15:53:03.966966 14521 raft_consensus.cc:2726] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260809 15:53:04.047603 14599 log.cc:924] Time spent T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.486s user 0.000s sys 0.002s
I20260809 15:53:04.127806 10893 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:03.942119 14374 consensus_queue.cc:237] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:04.061025 14529 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:04.065413 14527 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:53:04.116683 14537 consensus_meta.cc:287] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.694s user 0.009s sys 0.002s
I20260809 15:53:04.091491 14405 raft_consensus.cc:2741] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
I20260809 15:53:03.996008 13973 consensus_queue.cc:1035] T 93303d409ab940409d87013868417907 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.105s
I20260809 15:53:04.118108 11017 rpcz_store.cc:276] 0809 15:53:02.568270 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:02.568388 (+ 118us) service_pool.cc:227] Handling call
0809 15:53:02.568692 (+ 304us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:04.051717 (+1483025us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:04.073010 (+ 21293us) spinlock_profiling.cc:243] Waited 987 ms on lock 0x7b540019f5d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:04.073242 (+ 232us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2269672704}
W20260809 15:53:04.125053 10736 rpcz_store.cc:267] Call kudu.master.MasterService.TSHeartbeat from 127.0.0.1:33038 (request call id 393) took 22459 ms (client timeout 15000 ms). Trace:
I20260809 15:53:04.125186 14048 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:04.095657 14540 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:04.119714 11014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RunLeaderElection from 127.0.0.1:50542 (request call id 2484) took 1215 ms. Trace:
I20260809 15:53:04.160239 11014 rpcz_store.cc:276] 0809 15:53:02.892099 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:03.473567 (+581468us) service_pool.cc:227] Handling call
0809 15:53:04.107721 (+634154us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:53:04.122515 14436 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s user 0.003s sys 0.008s
I20260809 15:53:04.110643 14455 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:53:04.139341 14621 raft_consensus.cc:2781] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:53:04.158684 11632 raft_consensus.cc:2396] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 5 because replica has last-logged OpId of term: 4 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260809 15:53:03.974225 14394 consensus_queue.cc:1046] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 1.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.136s
I20260809 15:53:04.077297 14507 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:04.162042 14611 log.cc:924] Time spent T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.210s user 0.001s sys 0.000s
I20260809 15:53:04.163635 14048 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:04.162653 14613 log.cc:924] Time spent T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.335s user 0.001s sys 0.000s
I20260809 15:53:04.115849 14610 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:04.156958 10737 catalog_manager.cc:5495] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 4 to 5, leader changed from c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 5 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
W20260809 15:53:04.136248 14615 log.cc:924] Time spent T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.239s user 0.001s sys 0.000s
I20260809 15:53:04.164326 11009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2286) took 1300 ms. Trace:
W20260809 15:53:04.168510 11018 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.790s user 0.047s sys 0.131s
W20260809 15:53:04.200264 11018 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.822s user 0.047s sys 0.131s
I20260809 15:53:04.154584 14521 raft_consensus.cc:2726] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 15:53:04.184648 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:04.198622 11009 rpcz_store.cc:276] 0809 15:53:02.866408 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:03.416572 (+550164us) service_pool.cc:227] Handling call
0809 15:53:03.416901 (+ 329us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:03.417468 (+ 567us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:03.417502 (+ 34us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:03.417973 (+ 471us) log.cc:842] Serialized 26 byte log entry
0809 15:53:04.009250 (+591277us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:53:04.009305 (+ 55us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:04.009399 (+ 94us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:04.164119 (+154720us) raft_consensus.cc:1671] finished
0809 15:53:04.164137 (+ 18us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.164269 (+ 132us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":116,"raft.run_cpu_time_us":71,"raft.run_wall_time_us":72,"thread_start_us":590846,"threads_started":1,"wal-append.queue_time_us":677938}
I20260809 15:53:04.204084 11014 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
W20260809 15:53:04.162397 11605 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.504s user 0.002s sys 0.009s
W20260809 15:53:04.204833 11605 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.702s user 0.002s sys 0.009s
I20260809 15:53:04.208220 10926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 2111) took 2376 ms. Trace:
W20260809 15:53:04.165650 14348 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.760s user 0.029s sys 0.069s
I20260809 15:53:04.151542 14529 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:04.168351 14455 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
W20260809 15:53:04.161406 14463 consensus_meta.cc:287] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.955s user 0.012s sys 0.000s
I20260809 15:53:04.202445 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:04.242589 11017 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
W20260809 15:53:04.164826 11637 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.649s user 0.002s sys 0.015s
I20260809 15:53:04.208683 10926 rpcz_store.cc:276] 0809 15:53:01.821885 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:02.691223 (+869338us) service_pool.cc:227] Handling call
0809 15:53:02.691589 (+ 366us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:02.692089 (+ 500us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:02.692126 (+ 37us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:02.692633 (+ 507us) log.cc:842] Serialized 26 byte log entry
0809 15:53:03.243512 (+550879us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:53:03.243564 (+ 52us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:03.243656 (+ 92us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:04.198056 (+954400us) raft_consensus.cc:1671] finished
0809 15:53:04.198094 (+ 38us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.198262 (+ 168us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":425,"raft.run_cpu_time_us":51,"raft.run_wall_time_us":43,"thread_start_us":550404,"threads_started":1,"wal-append.queue_time_us":815258}
W20260809 15:53:04.192977 11634 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.511s user 0.013s sys 0.041s
I20260809 15:53:04.198280 14621 raft_consensus.cc:686] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [term 6 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:04.151214 14527 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:04.243455 11637 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.728s user 0.002s sys 0.015s
W20260809 15:53:04.243618 11634 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.586s user 0.014s sys 0.041s
I20260809 15:53:04.245357 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:04.187036 14540 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:53:04.240557 14348 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.836s user 0.030s sys 0.070s
I20260809 15:53:04.276239 11641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 1003) took 1711 ms. Trace:
I20260809 15:53:04.232077 14625 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:04.234835 14521 raft_consensus.cc:2781] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election won for term 5
W20260809 15:53:04.290203 11633 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.514s user 0.023s sys 0.031s
I20260809 15:53:04.190052 14619 consensus_queue.cc:1035] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.071s
W20260809 15:53:04.293726 11633 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.517s user 0.024s sys 0.031s
I20260809 15:53:04.292448 11634 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 1006) took 1718 ms. Trace:
I20260809 15:53:04.307610 10921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 2107) took 2494 ms. Trace:
I20260809 15:53:04.286551 11641 rpcz_store.cc:276] 0809 15:53:02.536731 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:02.536924 (+ 193us) service_pool.cc:227] Handling call
0809 15:53:02.549867 (+ 12943us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:02.558881 (+ 9014us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:02.558933 (+ 52us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:02.559557 (+ 624us) log.cc:842] Serialized 26 byte log entry
0809 15:53:02.985516 (+425959us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:53:02.985556 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:02.985626 (+ 70us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:03.502525 (+516899us) spinlock_profiling.cc:243] Waited 52.3 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:53:04.234296 (+731771us) raft_consensus.cc:1671] finished
0809 15:53:04.234323 (+ 27us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.247716 (+ 13393us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":3531,"raft.run_cpu_time_us":129,"raft.run_wall_time_us":129,"spinlock_wait_cycles":120195968,"thread_start_us":425493,"threads_started":1,"wal-append.queue_time_us":809794}
W20260809 15:53:04.280150 14620 log.cc:924] Time spent T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.289s user 0.001s sys 0.000s
I20260809 15:53:04.251542 14394 raft_consensus.cc:3032] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Stepping down as leader of term 3
I20260809 15:53:04.319514 11011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 763) took 1385 ms. Trace:
I20260809 15:53:04.279601 14407 raft_consensus.cc:2726] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260809 15:53:04.292742 14618 log.cc:924] Time spent T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.293s user 0.000s sys 0.001s
W20260809 15:53:04.328006 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 197235us
I20260809 15:53:04.197016 14610 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:04.329371 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:04.280546 14638 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:04.247800 14463 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:53:04.158354 10736 rpcz_store.cc:269] 0809 15:52:41.526889 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:52:41.527142 (+ 253us) service_pool.cc:227] Handling call
0809 15:53:03.980428 (+22453286us) inbound_call.cc:173] Queueing success response
Related trace 'op':
0809 15:52:53.336220 (+ 0us) write_op.cc:183] PREPARE: starting on tablet 00000000000000000000000000000000
0809 15:52:53.336847 (+ 627us) write_op.cc:432] Acquiring schema lock in shared mode
0809 15:52:53.336920 (+ 73us) write_op.cc:435] Acquired schema lock
0809 15:52:53.336938 (+ 18us) tablet.cc:660] Decoding operations
0809 15:52:53.340840 (+ 3902us) write_op.cc:620] Acquiring the partition lock for write op
0809 15:52:53.340929 (+ 89us) write_op.cc:641] Partition lock acquired for write op
0809 15:52:53.340952 (+ 23us) tablet.cc:683] Acquiring locks for 12 operations
0809 15:52:53.364734 (+ 23782us) tablet.cc:699] Row locks acquired
0809 15:52:53.372119 (+ 7385us) write_op.cc:260] PREPARE: finished
0809 15:52:53.435223 (+ 63104us) write_op.cc:270] Start()
0809 15:52:53.461674 (+ 26451us) write_op.cc:276] Timestamp: P: 1786290773410520 usec, L: 0
0809 15:52:53.484956 (+ 23282us) op_driver.cc:331] REPLICATION: starting
0809 15:52:53.528998 (+ 44042us) log.cc:842] Serialized 4657 byte log entry
0809 15:52:53.912378 (+383380us) op_driver.cc:444] REPLICATION: finished
0809 15:52:54.176507 (+264129us) write_op.cc:301] APPLY: starting
0809 15:52:54.189457 (+ 12950us) tablet.cc:1365] starting BulkCheckPresence
0809 15:52:54.286419 (+ 96962us) tablet.cc:1368] finished BulkCheckPresence
0809 15:52:54.315061 (+ 28642us) tablet.cc:1370] starting ApplyRowOperation cycle
0809 15:52:54.564664 (+249603us) tablet.cc:1381] finished ApplyRowOperation cycle
0809 15:52:54.565035 (+ 371us) tablet_metrics.cc:531] ProbeStats: bloom_lookups=0,key_file_lookups=0,delta_file_lookups=0,mrs_lookups=12
0809 15:52:54.565084 (+ 49us) write_op.cc:312] APPLY: finished
0809 15:52:54.565486 (+ 402us) log.cc:842] Serialized 89 byte log entry
0809 15:52:54.565647 (+ 161us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:52:54.565790 (+ 143us) write_op.cc:454] Released schema lock
0809 15:52:54.565877 (+ 87us) write_op.cc:341] FINISH: Updating metrics
0809 15:52:54.566231 (+ 354us) write_op.cc:489] Releasing partition, row and schema locks
0809 15:52:54.566265 (+ 34us) write_op.cc:454] Released schema lock
Metrics: {"mutex_wait_us":26261,"reported_tablets":34,"child_traces":[["op",{"apply.queue_time_us":228780,"apply.run_cpu_time_us":7090,"apply.run_wall_time_us":414365,"num_ops":12,"prepare.queue_time_us":146579,"prepare.run_cpu_time_us":12427,"prepare.run_wall_time_us":238990,"raft.queue_time_us":222261,"raft.run_cpu_time_us":4378,"raft.run_wall_time_us":89173,"replication_time_us":414825,"thread_start_us":269817,"threads_started":4,"wal-append.queue_time_us":117038,"wal-append.run_cpu_time_us":74423,"wal-append.run_wall_time_us":1930915}]]}
I20260809 15:53:04.327665 11011 rpcz_store.cc:276] 0809 15:53:02.922290 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:03.910940 (+988650us) service_pool.cc:227] Handling call
0809 15:53:03.911255 (+ 315us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:03.913314 (+ 2059us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:03.913343 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:03.913780 (+ 437us) log.cc:842] Serialized 26 byte log entry
0809 15:53:04.055281 (+141501us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:53:04.055331 (+ 50us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:04.055403 (+ 72us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:04.307699 (+252296us) raft_consensus.cc:1671] finished
0809 15:53:04.307725 (+ 26us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.307876 (+ 151us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":1423,"raft.queue_time_us":1533,"raft.run_cpu_time_us":76,"raft.run_wall_time_us":77,"thread_start_us":141113,"threads_started":1,"wal-append.queue_time_us":196344}
I20260809 15:53:04.308023 10921 rpcz_store.cc:276] 0809 15:53:01.813455 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:02.688100 (+874645us) service_pool.cc:227] Handling call
0809 15:53:02.688501 (+ 401us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:02.688971 (+ 470us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:02.689010 (+ 39us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:02.695481 (+ 6471us) log.cc:842] Serialized 26 byte log entry
0809 15:53:03.273845 (+578364us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:53:03.273911 (+ 66us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:03.275903 (+ 1992us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:04.307370 (+1031467us) raft_consensus.cc:1671] finished
0809 15:53:04.307390 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.307528 (+ 138us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":3290,"raft.run_cpu_time_us":132,"raft.run_wall_time_us":133,"thread_start_us":514057,"threads_started":1,"wal-append.queue_time_us":810633}
I20260809 15:53:04.499521 11632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41482 (request call id 784) took 1228 ms. Trace:
W20260809 15:53:04.543848 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 383219us
I20260809 15:53:04.613410 11639 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 801) took 1197 ms. Trace:
I20260809 15:53:04.306051 11634 rpcz_store.cc:276] 0809 15:53:02.574048 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:02.582196 (+ 8148us) service_pool.cc:227] Handling call
0809 15:53:02.582570 (+ 374us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 15:53:02.583065 (+ 495us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:02.583090 (+ 25us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 15:53:02.583637 (+ 547us) log.cc:842] Serialized 52 byte log entry
0809 15:53:03.089411 (+505774us) raft_consensus.cc:1631] Marking committed up to 8
0809 15:53:03.426896 (+337485us) spinlock_profiling.cc:243] Waited 24.7 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:53:03.657313 (+230417us) log.cc:842] Serialized 14 byte log entry
0809 15:53:04.247849 (+590536us) log.cc:842] Serialized 14 byte log entry
0809 15:53:04.280229 (+ 32380us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:04.280296 (+ 67us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:04.292196 (+ 11900us) raft_consensus.cc:1671] finished
0809 15:53:04.292217 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.292383 (+ 166us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":32137,"prepare.queue_time_us":199007,"prepare.run_cpu_time_us":70,"prepare.run_wall_time_us":86,"raft.queue_time_us":327087,"raft.run_cpu_time_us":254,"raft.run_wall_time_us":2994,"spinlock_wait_cycles":56894848,"thread_start_us":1532091,"threads_started":3,"wal-append.queue_time_us":691574}
W20260809 15:53:04.332260 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 203229us
I20260809 15:53:04.595873 11640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 800) took 1196 ms. Trace:
I20260809 15:53:04.666667 11009 raft_consensus.cc:2379] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 7: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:53:04.311259 10737 catalog_manager.cc:5495] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 2 to 5, leader changed from d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 5 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } }
I20260809 15:53:04.663892 11632 rpcz_store.cc:276] 0809 15:53:03.271100 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:03.507934 (+236834us) service_pool.cc:227] Handling call
0809 15:53:04.498160 (+990226us) spinlock_profiling.cc:243] Waited 624 ms on lock 0x7b5400694bd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:04.499425 (+ 1265us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1434304384}
I20260809 15:53:04.668673 10978 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:04.346899 11014 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 8.
I20260809 15:53:04.677078 10978 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:04.670849 10809 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:53:04.556113 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 439035us
I20260809 15:53:04.696480 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
W20260809 15:53:04.515964 14623 log.cc:924] Time spent T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.079s user 0.000s sys 0.000s
W20260809 15:53:04.516594 14631 log.cc:924] Time spent T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.135s user 0.000s sys 0.000s
W20260809 15:53:04.630673 14005 consensus_meta.cc:287] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 1.162s user 0.014s sys 0.000s
W20260809 15:53:04.516806 14632 log.cc:924] Time spent T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.309s user 0.000s sys 0.001s
W20260809 15:53:04.530650 14402 consensus_meta.cc:287] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 1.093s user 0.013s sys 0.002s
I20260809 15:53:04.722905 11637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 1007) took 2109 ms. Trace:
I20260809 15:53:04.723304 11637 rpcz_store.cc:276] 0809 15:53:02.613851 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:02.613976 (+ 125us) service_pool.cc:227] Handling call
0809 15:53:02.616724 (+ 2748us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:02.617186 (+ 462us) raft_consensus.cc:1520] Early marking committed up to index 5
0809 15:53:03.093160 (+475974us) log.cc:842] Serialized 14 byte log entry
0809 15:53:04.244820 (+1151660us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:04.278168 (+ 33348us) log.cc:842] Serialized 26 byte log entry
0809 15:53:04.291178 (+ 13010us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:53:04.310138 (+ 18960us) log.cc:842] Serialized 14 byte log entry
0809 15:53:04.310333 (+ 195us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:04.310415 (+ 82us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:04.722665 (+412250us) raft_consensus.cc:1671] finished
0809 15:53:04.722687 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.722812 (+ 125us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":5834,"prepare.queue_time_us":689506,"prepare.run_cpu_time_us":78,"prepare.run_wall_time_us":82,"raft.queue_time_us":634597,"raft.run_cpu_time_us":422,"raft.run_wall_time_us":33803,"thread_start_us":1604726,"threads_started":3,"wal-append.queue_time_us":742373}
I20260809 15:53:04.057737 11016 rpcz_store.cc:276] 0809 15:53:02.891430 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:03.434410 (+542980us) service_pool.cc:227] Handling call
0809 15:53:03.434728 (+ 318us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:03.435027 (+ 299us) raft_consensus.cc:1520] Early marking committed up to index 6
0809 15:53:03.435332 (+ 305us) log.cc:842] Serialized 14 byte log entry
0809 15:53:04.056760 (+621428us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:04.056827 (+ 67us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:53:04.056900 (+ 73us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:04.056979 (+ 79us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.057119 (+ 140us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":2884,"prepare.run_cpu_time_us":41,"prepare.run_wall_time_us":41,"raft.queue_time_us":9289,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":33651,"thread_start_us":603694,"threads_started":1}
I20260809 15:53:04.728410 11016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2288) took 1652 ms. Trace:
I20260809 15:53:04.728704 11016 rpcz_store.cc:276] 0809 15:53:03.075652 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:04.727706 (+1652054us) service_pool.cc:227] Handling call
0809 15:53:04.728005 (+ 299us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:04.728152 (+ 147us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:04.728175 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:04.728209 (+ 34us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:53:04.728237 (+ 28us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:04.728280 (+ 43us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.728361 (+ 81us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:53:04.654525 14436 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:04.665831 11640 rpcz_store.cc:276] 0809 15:53:03.399738 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:03.924398 (+524660us) service_pool.cc:227] Handling call
0809 15:53:03.924696 (+ 298us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:04.204430 (+279734us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:04.204469 (+ 39us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:04.204905 (+ 436us) log.cc:842] Serialized 26 byte log entry
0809 15:53:04.430842 (+225937us) raft_consensus.cc:1631] Marking committed up to 7
0809 15:53:04.434471 (+ 3629us) log.cc:842] Serialized 14 byte log entry
0809 15:53:04.434822 (+ 351us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:04.434962 (+ 140us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:04.595635 (+160673us) raft_consensus.cc:1671] finished
0809 15:53:04.595655 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.595797 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":3161,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":241866,"raft.run_cpu_time_us":179,"raft.run_wall_time_us":26345,"spinlock_wait_cycles":49276800,"thread_start_us":426865,"threads_started":2,"wal-append.queue_time_us":380940}
W20260809 15:53:04.628048 14633 log.cc:924] Time spent T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.096s user 0.000s sys 0.000s
W20260809 15:53:04.447449 14628 log.cc:924] Time spent T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.256s user 0.000s sys 0.001s
W20260809 15:53:04.448220 14627 log.cc:924] Time spent T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.201s user 0.000s sys 0.000s
W20260809 15:53:04.542802 14636 log.cc:924] Time spent T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.110s user 0.000s sys 0.000s
I20260809 15:53:04.314052 14639 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:53:04.515735 14617 log.cc:924] Time spent T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.238s user 0.000s sys 0.000s
I20260809 15:53:04.693795 14048 raft_consensus.cc:2781] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 15:53:04.709645 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:04.747342 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:04.331079 14521 raft_consensus.cc:686] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:04.532027 14507 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:04.698159 14658 raft_consensus.cc:2781] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:53:04.306329 14537 consensus_queue.cc:1035] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.388s
I20260809 15:53:04.692622 14638 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:04.530808 14648 consensus_queue.cc:1046] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 8.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.429s
I20260809 15:53:04.748039 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:04.756625 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:04.713207 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:04.714502 11015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 756) took 1986 ms. Trace:
W20260809 15:53:04.715236 14645 log.cc:924] Time spent T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.181s user 0.001s sys 0.000s
I20260809 15:53:04.715524 14348 consensus_queue.cc:1035] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 15:53:04.615547 14657 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:04.742308 10737 catalog_manager.cc:5495] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 5 to 6, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3), no longer has a pending config: VOTER a53441872c0c416b8e2f26f52399f536 (127.9.84.1), VOTER 20730d75ad954a82b126148f8152a9f3 (127.9.84.2), VOTER d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3), VOTER c7aa762318cb4e97875333f10679a9f9 (127.9.84.4). New cstate: current_term: 6 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: NON_VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: true } health_report { overall_health: UNKNOWN } } }
W20260809 15:53:04.449334 14622 log.cc:924] Time spent T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.245s user 0.000s sys 0.000s
I20260809 15:53:04.421365 14640 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:53:04.431572 14616 log.cc:924] Time spent T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.394s user 0.000s sys 0.001s
I20260809 15:53:04.451159 14540 leader_election.cc:288] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:04.434113 14624 raft_consensus.cc:482] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:53:04.448732 14637 log.cc:924] Time spent T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.240s user 0.000s sys 0.000s
I20260809 15:53:04.295787 14625 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:04.327443 14394 raft_consensus.cc:729] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:04.721715 14005 raft_consensus.cc:2726] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260809 15:53:04.449966 14629 log.cc:924] Time spent T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.203s user 0.000s sys 0.000s
I20260809 15:53:04.731489 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:04.721577 14529 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:04.804194 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:04.804534 11010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2272) took 2210 ms. Trace:
I20260809 15:53:04.804999 10843 raft_consensus.cc:2379] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 7: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:53:04.804934 11010 rpcz_store.cc:276] 0809 15:53:02.594361 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:02.695660 (+101299us) service_pool.cc:227] Handling call
0809 15:53:02.727443 (+ 31783us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:03.559763 (+832320us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:03.559803 (+ 40us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:03.738327 (+178524us) log.cc:842] Serialized 26 byte log entry
0809 15:53:04.234425 (+496098us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:53:04.270681 (+ 36256us) log.cc:842] Serialized 14 byte log entry
0809 15:53:04.270938 (+ 257us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:04.271039 (+ 101us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:04.801002 (+529963us) spinlock_profiling.cc:243] Waited 4.72 ms on lock 0x7b1800036c40. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f79665cb638 00007f79665ceb11 00007f79665ceac9 00007f79665cea59 00007f79665cea21 00007f79665cdd1d 00007f796f82f204 00007f796f82f039 00007f79665addc6
0809 15:53:04.803271 (+ 2269us) spinlock_profiling.cc:243] Waited 12.6 ms on lock 0x7b1800036c40. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f79665cb638 00007f79665ceb11 00007f79665ceac9 00007f79665cea59 00007f79665cea21 00007f79665cdd1d 00007f796f82f204 00007f796f82f039 00007f79665addc6
0809 15:53:04.804304 (+ 1033us) raft_consensus.cc:1671] finished
0809 15:53:04.804324 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.804453 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":41915,"prepare.run_cpu_time_us":2217,"prepare.run_wall_time_us":16714,"raft.queue_time_us":39017,"raft.run_cpu_time_us":226,"raft.run_wall_time_us":228,"spinlock_wait_cycles":39909888,"thread_start_us":1107427,"threads_started":2,"wal-append.queue_time_us":593477}
I20260809 15:53:04.807169 11011 raft_consensus.cc:1226] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 4. Current term is 5. Ops: []
I20260809 15:53:04.809136 11012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2489) took 1915 ms. Trace:
I20260809 15:53:04.628901 14527 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:04.751556 14639 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:04.809482 11012 rpcz_store.cc:276] 0809 15:53:02.893729 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:03.680624 (+786895us) service_pool.cc:227] Handling call
0809 15:53:03.742476 (+ 61852us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:03.912152 (+169676us) raft_consensus.cc:1520] Early marking committed up to index 7
0809 15:53:03.912480 (+ 328us) log.cc:842] Serialized 14 byte log entry
0809 15:53:04.165319 (+252839us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:04.193319 (+ 28000us) log.cc:842] Serialized 26 byte log entry
0809 15:53:04.193470 (+ 151us) raft_consensus.cc:1631] Marking committed up to 7
0809 15:53:04.193510 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:04.193591 (+ 81us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:04.808908 (+615317us) raft_consensus.cc:1671] finished
0809 15:53:04.808929 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.809072 (+ 143us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":137195,"prepare.queue_time_us":162,"prepare.run_cpu_time_us":47,"prepare.run_wall_time_us":44,"raft.queue_time_us":159542,"raft.run_cpu_time_us":223,"raft.run_wall_time_us":6319,"thread_start_us":252003,"threads_started":1,"wal-append.queue_time_us":532626}
I20260809 15:53:04.810891 11018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 757) took 2077 ms. Trace:
I20260809 15:53:04.811275 11018 rpcz_store.cc:276] 0809 15:53:02.733827 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:03.189669 (+455842us) service_pool.cc:227] Handling call
0809 15:53:03.204141 (+ 14472us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:03.225372 (+ 21231us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:03.225416 (+ 44us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:03.288264 (+ 62848us) log.cc:842] Serialized 26 byte log entry
0809 15:53:04.200701 (+912437us) raft_consensus.cc:1631] Marking committed up to 7
0809 15:53:04.200762 (+ 61us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:04.200844 (+ 82us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:04.810696 (+609852us) raft_consensus.cc:1671] finished
0809 15:53:04.810714 (+ 18us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.810842 (+ 128us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":23023,"raft.run_cpu_time_us":160,"raft.run_wall_time_us":9174,"thread_start_us":822677,"threads_started":1,"wal-append.queue_time_us":1119990}
I20260809 15:53:04.812366 14639 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:04.664541 11639 rpcz_store.cc:276] 0809 15:53:03.400131 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:03.946425 (+546294us) service_pool.cc:227] Handling call
0809 15:53:03.946802 (+ 377us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:04.251957 (+305155us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:04.271434 (+ 19477us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:04.409441 (+138007us) log.cc:842] Serialized 26 byte log entry
0809 15:53:04.526814 (+117373us) raft_consensus.cc:1631] Marking committed up to 7
0809 15:53:04.547750 (+ 20936us) log.cc:842] Serialized 14 byte log entry
0809 15:53:04.547976 (+ 226us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:04.548064 (+ 88us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:04.597024 (+ 48960us) raft_consensus.cc:1671] finished
0809 15:53:04.597130 (+ 106us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.597289 (+ 159us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":15767,"prepare.queue_time_us":20658,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":42,"raft.queue_time_us":212357,"raft.run_cpu_time_us":261,"raft.run_wall_time_us":48630,"thread_start_us":376115,"threads_started":2,"wal-append.queue_time_us":174480}
I20260809 15:53:04.737186 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:04.737830 11016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 768) took 1306 ms. Trace:
I20260809 15:53:04.742427 11017 raft_consensus.cc:2454] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:53:04.743456 11009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 764) took 1819 ms. Trace:
I20260809 15:53:04.746951 11014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2287) took 1682 ms. Trace:
I20260809 15:53:04.815434 11014 rpcz_store.cc:276] 0809 15:53:03.064419 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:04.734791 (+1670372us) service_pool.cc:227] Handling call
0809 15:53:04.745613 (+ 10822us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:04.745803 (+ 190us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:04.745837 (+ 34us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:04.746641 (+ 804us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:53:04.746689 (+ 48us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:04.746754 (+ 65us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.746873 (+ 119us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:53:04.815234 10737 catalog_manager.cc:5495] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 4 to 5, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 5 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
I20260809 15:53:04.443781 14621 consensus_queue.cc:237] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:04.750294 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:04.750432 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:04.750897 10838 raft_consensus.cc:2379] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 6: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:04.734347 10893 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:04.755939 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:04.753268 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:53:04.739535 10811 connection.cc:232] RPC call timeout handler was delayed by -1.2706305919564329s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
I20260809 15:53:04.762420 11015 rpcz_store.cc:276] 0809 15:53:02.727717 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:03.136049 (+408332us) service_pool.cc:227] Handling call
0809 15:53:03.136330 (+ 281us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:03.147285 (+ 10955us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:03.147314 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:03.171550 (+ 24236us) log.cc:842] Serialized 26 byte log entry
0809 15:53:03.966439 (+794889us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:53:03.966482 (+ 43us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:03.966556 (+ 74us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:04.714281 (+747725us) raft_consensus.cc:1671] finished
0809 15:53:04.714313 (+ 32us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.714447 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":30020,"raft.run_cpu_time_us":147,"raft.run_wall_time_us":5932,"thread_start_us":664593,"threads_started":1,"wal-append.queue_time_us":1022610}
I20260809 15:53:04.770349 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:04.818332 10844 raft_consensus.cc:2379] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 7: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:04.756699 14537 consensus_queue.cc:1225] T b6d399a7394c4a59b90a308aedfdf899 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer 20730d75ad954a82b126148f8152a9f3 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260809 15:53:04.767726 14473 consensus_meta.cc:287] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 1.289s user 0.012s sys 0.002s
I20260809 15:53:04.780946 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:04.753878 14658 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:04.782795 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:04.835923 10927 raft_consensus.cc:2379] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 7: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:04.836319 10923 raft_consensus.cc:2379] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 7: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:04.785005 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:53:04.836828 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 102532us
I20260809 15:53:04.837204 10921 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 4.
I20260809 15:53:04.837831 10893 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:04.786441 11636 raft_consensus.cc:1226] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260809 15:53:04.788102 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:53:04.799655 11606 connection.cc:232] RPC call timeout handler was delayed by -1.1669618029845878s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16890000000000002s for a response
W20260809 15:53:04.805281 14455 consensus_meta.cc:287] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.512s user 0.006s sys 0.002s
I20260809 15:53:04.737275 14436 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:04.843950 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:04.757164 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:04.814872 11009 rpcz_store.cc:276] 0809 15:53:02.924347 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:04.698500 (+1774153us) service_pool.cc:227] Handling call
0809 15:53:04.743163 (+ 44663us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:04.743287 (+ 124us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:04.743299 (+ 12us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:04.743321 (+ 22us) raft_consensus.cc:1631] Marking committed up to 9
0809 15:53:04.743335 (+ 14us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:04.743364 (+ 29us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.743418 (+ 54us) inbound_call.cc:173] Queueing success response
Metrics: {}
W20260809 15:53:04.818257 10811 connection.cc:232] RPC call timeout handler was delayed by -1.1573553599882871s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15700000000000003s for a response
I20260809 15:53:04.845366 10837 raft_consensus.cc:2396] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 8 because replica has last-logged OpId of term: 7 index: 9, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
W20260809 15:53:04.845423 10811 connection.cc:232] RPC call timeout handler was delayed by -1.0140477499808185s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1526s for a response
I20260809 15:53:04.817631 11016 rpcz_store.cc:276] 0809 15:53:03.426560 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:04.729512 (+1302952us) service_pool.cc:227] Handling call
0809 15:53:04.729750 (+ 238us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:04.729912 (+ 162us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:04.729935 (+ 23us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:04.729968 (+ 33us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:53:04.729994 (+ 26us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:04.730037 (+ 43us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.732412 (+ 2375us) inbound_call.cc:173] Queueing success response
Metrics: {}
I20260809 15:53:04.821833 11642 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 9.
I20260809 15:53:04.834156 10931 raft_consensus.cc:2379] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 7: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:04.763726 14048 raft_consensus.cc:686] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 8 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
W20260809 15:53:04.839474 11606 connection.cc:232] RPC call timeout handler was delayed by -1.1048618030035868s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1758s for a response
I20260809 15:53:04.800115 14624 raft_consensus.cc:504] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:04.794883 14561 consensus_meta.cc:287] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 1.258s user 0.014s sys 0.000s
I20260809 15:53:04.836882 10920 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 2120) took 2297 ms. Trace:
I20260809 15:53:04.787132 14540 raft_consensus.cc:2781] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:53:04.795171 14402 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
W20260809 15:53:04.842417 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 165410us
I20260809 15:53:04.846465 11013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2476) took 2405 ms. Trace:
I20260809 15:53:04.852609 10893 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:04.818284 10847 raft_consensus.cc:2454] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:53:04.855650 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:04.855444 11606 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:04.863770 10893 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:04.864873 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:04.866343 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 1 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:04.868854 11010 raft_consensus.cc:1226] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4. Ops: []
I20260809 15:53:04.840065 14463 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:04.847012 14473 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:04.870455 14048 consensus_queue.cc:237] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:04.843236 14527 consensus_queue.cc:1046] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 4.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
I20260809 15:53:04.871874 14473 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:04.872447 14527 raft_consensus.cc:3032] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:53:04.872792 14527 raft_consensus.cc:729] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:04.856065 14561 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:04.874189 14394 consensus_queue.cc:260] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:04.857714 11013 rpcz_store.cc:276] 0809 15:53:02.441133 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:02.898198 (+457065us) service_pool.cc:227] Handling call
0809 15:53:02.898586 (+ 388us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:02.899003 (+ 417us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:02.899035 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:02.899580 (+ 545us) log.cc:842] Serialized 26 byte log entry
0809 15:53:03.892091 (+992511us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:53:03.892131 (+ 40us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:03.892200 (+ 69us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:04.846269 (+954069us) raft_consensus.cc:1671] finished
0809 15:53:04.846289 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.846413 (+ 124us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":7812,"raft.run_cpu_time_us":168,"raft.run_wall_time_us":14105,"thread_start_us":992108,"threads_started":1,"wal-append.queue_time_us":1295409}
I20260809 15:53:04.875281 14394 raft_consensus.cc:3037] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:53:04.876212 11015 raft_consensus.cc:1261] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260809 15:53:04.876101 14561 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:04.878443 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:04.871266 14436 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:04.840128 14652 raft_consensus.cc:2726] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:53:04.772130 14657 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:04.893262 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 127676us
I20260809 15:53:04.878005 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:04.841369 14658 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:04.900509 10927 raft_consensus.cc:2379] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 4: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
W20260809 15:53:04.879097 14662 log.cc:924] Time spent T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.081s user 0.000s sys 0.000s
I20260809 15:53:04.902851 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:04.905525 11633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 1012) took 1888 ms. Trace:
I20260809 15:53:04.882514 14655 raft_consensus.cc:2726] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260809 15:53:04.784621 14640 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:04.861056 14503 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:04.856313 14540 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:04.905829 11633 rpcz_store.cc:276] 0809 15:53:03.017135 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:03.556117 (+538982us) service_pool.cc:227] Handling call
0809 15:53:03.743099 (+186982us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 15:53:04.307852 (+564753us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:04.308290 (+ 438us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 15:53:04.567068 (+258778us) log.cc:842] Serialized 52 byte log entry
0809 15:53:04.781703 (+214635us) raft_consensus.cc:1631] Marking committed up to 7
0809 15:53:04.788787 (+ 7084us) log.cc:842] Serialized 14 byte log entry
0809 15:53:04.789002 (+ 215us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:04.789115 (+ 113us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:04.806484 (+ 17369us) spinlock_profiling.cc:243] Waited 4.67 ms on lock 0x7b18002bce40. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f79665cb764 00007f79665ceb11 00007f79665ceac9 00007f79665cea59 00007f79665cea21 00007f79665cdd1d 00007f796f82f204 00007f796f82f039 00007f79665addc6
0809 15:53:04.881086 (+ 74602us) raft_consensus.cc:1671] finished
0809 15:53:04.905318 (+ 24232us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.905462 (+ 144us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":12836,"prepare.run_cpu_time_us":42,"prepare.run_wall_time_us":43,"raft.queue_time_us":254977,"raft.run_cpu_time_us":148,"raft.run_wall_time_us":7635,"spinlock_wait_cycles":17203840,"thread_start_us":662449,"threads_started":2,"wal-append.queue_time_us":252812}
I20260809 15:53:04.917973 10891 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:04.918562 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 10 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:04.919744 10921 raft_consensus.cc:3037] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:04.873612 14527 consensus_queue.cc:260] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:04.927045 14527 raft_consensus.cc:3037] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:53:04.878244 14624 leader_election.cc:288] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:04.877929 14610 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:04.906243 14654 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:04.928607 14654 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:53:04.929587 11642 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41478 (request call id 1059) took 2137 ms (client timeout 2805 ms). Trace:
I20260809 15:53:04.897931 14473 raft_consensus.cc:2726] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260809 15:53:04.922098 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 70874.7us
I20260809 15:53:04.903611 14561 raft_consensus.cc:2781] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:04.908854 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:04.904248 14348 consensus_queue.cc:1035] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.020s
I20260809 15:53:04.908520 14612 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:04.916755 13973 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:04.924817 14625 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:04.843740 14455 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:04.862819 14521 consensus_queue.cc:237] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:04.930075 11642 rpcz_store.cc:269] 0809 15:53:02.792637 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:03.455856 (+663219us) service_pool.cc:227] Handling call
0809 15:53:04.929346 (+1473490us) spinlock_profiling.cc:243] Waited 1.31 s on lock 0x7b540083b154. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:04.929524 (+ 178us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3006870784}
I20260809 15:53:04.883728 14402 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:04.937944 10737 catalog_manager.cc:5495] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 3 to 6. New cstate: current_term: 6 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } }
W20260809 15:53:04.943884 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 66336.8us
W20260809 15:53:04.944470 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 48103.6us
I20260809 15:53:04.922278 14540 raft_consensus.cc:3037] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:53:04.950505 10923 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:53:04.930998 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:04.955700 10923 raft_consensus.cc:595] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:04.945950 14655 raft_consensus.cc:2781] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:53:04.943301 14561 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:04.956185 10923 raft_consensus.cc:603] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 15:53:04.932621 14254 raft_consensus.cc:2726] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:53:04.965291 10930 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:04.965961 10737 catalog_manager.cc:5495] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 2 to 5, leader changed from 20730d75ad954a82b126148f8152a9f3 (127.9.84.2) to a53441872c0c416b8e2f26f52399f536 (127.9.84.1). New cstate: current_term: 5 leader_uuid: "a53441872c0c416b8e2f26f52399f536" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } }
I20260809 15:53:04.932164 14503 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:04.849431 10920 rpcz_store.cc:276] 0809 15:53:02.539580 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:03.286879 (+747299us) service_pool.cc:227] Handling call
0809 15:53:03.287214 (+ 335us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:03.303412 (+ 16198us) raft_consensus.cc:1520] Early marking committed up to index 7
0809 15:53:03.369019 (+ 65607us) log.cc:842] Serialized 14 byte log entry
0809 15:53:04.824411 (+1455392us) spinlock_profiling.cc:243] Waited 12.8 ms on lock 0x7b180035e6c0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f796918282c 00007f79665ad9dc 00007f796f3e4a39 00007f79665ca8bb 00007f79665c8cbf 00007f79665cae22 00007f79691745bb 00007f796917a063 00007f796917a57b
0809 15:53:04.836598 (+ 12187us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:04.836647 (+ 49us) raft_consensus.cc:1631] Marking committed up to 7
0809 15:53:04.836688 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:04.836745 (+ 57us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:04.836833 (+ 88us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":11944,"prepare.queue_time_us":34192,"prepare.run_cpu_time_us":48,"prepare.run_wall_time_us":49,"raft.queue_time_us":12624,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":8855,"spinlock_wait_cycles":29505920,"thread_start_us":667366,"threads_started":1,"wal-append.queue_time_us":669003}
I20260809 15:53:05.007047 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:05.007560 10837 raft_consensus.cc:1226] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 7. Current term is 8. Ops: [7.5-7.5]
W20260809 15:53:05.005913 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 142226us
I20260809 15:53:05.009097 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:05.010473 10893 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:53:05.014997 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 54815.1us
I20260809 15:53:04.962661 14561 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:04.965323 14655 raft_consensus.cc:686] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:04.950354 13973 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:53:05.032510 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 77383.5us
I20260809 15:53:05.034617 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:04.950242 14612 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:05.041213 10811 leader_election.cc:334] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.580s (SENT)
I20260809 15:53:05.041940 10811 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:05.043187 14672 raft_consensus.cc:2726] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260809 15:53:05.044636 10811 leader_election.cc:334] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.781s (SENT)
I20260809 15:53:05.034803 10848 raft_consensus.cc:2454] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:53:04.962724 14625 consensus_queue.cc:1046] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260809 15:53:05.004899 14254 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:05.036216 10737 catalog_manager.cc:5495] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 3 to 4, leader changed from c7aa762318cb4e97875333f10679a9f9 (127.9.84.4) to d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3). New cstate: current_term: 4 leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" committed_config { opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } health_report { overall_health: UNKNOWN } } }
I20260809 15:53:05.061043 10930 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 8.
W20260809 15:53:05.061275 10978 connection.cc:232] RPC call timeout handler was delayed by -2.2766312780440785s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15980000000000003s for a response
I20260809 15:53:05.038450 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:05.052294 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:05.054296 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:05.054709 11018 raft_consensus.cc:1261] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 1 index: 4. Preceding OpId from leader: term: 4 index: 5. (index mismatch)
I20260809 15:53:05.063645 10841 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Advancing to term 10
W20260809 15:53:05.065372 10978 connection.cc:232] RPC call timeout handler was delayed by -2.0561312779900618s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18430000000000002s for a response
W20260809 15:53:05.065763 10978 connection.cc:232] RPC call timeout handler was delayed by -2.0300312780309469s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18720000000000003s for a response
I20260809 15:53:05.055151 14666 raft_consensus.cc:2726] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260809 15:53:05.066160 10978 connection.cc:232] RPC call timeout handler was delayed by -2.0300312780309469s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18720000000000003s for a response
I20260809 15:53:05.066860 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:53:05.069464 10978 connection.cc:232] RPC call timeout handler was delayed by -1.9857762730098329s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18560000000000001s for a response
W20260809 15:53:05.070247 10978 connection.cc:232] RPC call timeout handler was delayed by -1.9544312779908068s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1956s for a response
W20260809 15:53:05.070533 10978 connection.cc:232] RPC call timeout handler was delayed by -1.8581312780152075s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20630000000000004s for a response
W20260809 15:53:05.070819 10978 connection.cc:232] RPC call timeout handler was delayed by -1.8257312780478969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
W20260809 15:53:05.071173 10978 connection.cc:232] RPC call timeout handler was delayed by -1.8257312780478969s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.20990000000000003s for a response
W20260809 15:53:05.071417 10978 connection.cc:232] RPC call timeout handler was delayed by -1.7240312779904343s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22120000000000004s for a response
W20260809 15:53:05.071664 10978 connection.cc:232] RPC call timeout handler was delayed by -1.7240312779904343s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.22120000000000004s for a response
I20260809 15:53:05.073001 14503 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:05.072566 14655 consensus_queue.cc:237] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 4.2, Last appended by leader: 2, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:05.075436 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:05.076159 10840 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:53:05.082836 10737 catalog_manager.cc:5495] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa reported cstate change: term changed from 3 to 4, leader changed from a53441872c0c416b8e2f26f52399f536 (127.9.84.1) to 20730d75ad954a82b126148f8152a9f3 (127.9.84.2). New cstate: current_term: 4 leader_uuid: "20730d75ad954a82b126148f8152a9f3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } }
I20260809 15:53:05.080025 10811 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:05.100598 14612 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:05.101178 14612 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:05.101938 13973 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:05.080812 14625 raft_consensus.cc:3032] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Stepping down as leader of term 3
W20260809 15:53:05.105155 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 85782.4us
I20260809 15:53:05.107277 10811 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
W20260809 15:53:05.109285 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 150792us
I20260809 15:53:05.068286 14254 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:05.101655 14612 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:05.115384 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:05.116418 11014 raft_consensus.cc:2379] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 6: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:05.094971 14672 raft_consensus.cc:2726] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:05.119340 11606 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:53:05.119668 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:05.120280 11014 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 8.
I20260809 15:53:05.105476 14625 raft_consensus.cc:729] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 3 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:05.088792 14676 consensus_queue.cc:1035] T 974451500a6e434f8d407d9f178ab710 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.091s
I20260809 15:53:05.111841 13973 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:53:05.175414 10891 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:05.318607 10841 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 10.
W20260809 15:53:05.369917 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 194582us
I20260809 15:53:05.255046 10921 raft_consensus.cc:2454] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:53:05.180027 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:05.111123 14654 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:05.182441 10924 raft_consensus.cc:2396] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 4.
I20260809 15:53:05.366911 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:05.384961 11014 raft_consensus.cc:2354] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8.
W20260809 15:53:05.327088 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 208401us
I20260809 15:53:05.125700 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:05.342365 10847 raft_consensus.cc:2454] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:53:05.118211 14503 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:53:05.176575 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 72423.1us
I20260809 15:53:05.208635 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260809 15:53:05.402913 11606 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:53:05.214593 14634 log.cc:924] Time spent T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.759s user 0.004s sys 0.000s
I20260809 15:53:05.409745 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:05.122118 14657 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:05.220108 14670 log.cc:924] Time spent T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.196s user 0.000s sys 0.001s
W20260809 15:53:05.219429 14671 log.cc:924] Time spent T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.143s user 0.000s sys 0.001s
I20260809 15:53:05.425060 10891 leader_election.cc:398] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Vote denied by peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223) with higher term. Message: Invalid argument: T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8.
I20260809 15:53:05.425488 10891 leader_election.cc:401] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term
I20260809 15:53:05.174854 14652 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:05.426215 14652 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:05.428956 14652 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:05.431252 11012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 769) took 1390 ms. Trace:
I20260809 15:53:05.431720 11012 rpcz_store.cc:276] 0809 15:53:04.040521 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:04.811244 (+770723us) service_pool.cc:227] Handling call
0809 15:53:04.950734 (+139490us) spinlock_profiling.cc:243] Waited 11.6 ms on lock 0x7b5800027098. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f7966593851 00007f7966593702 00007f796f483b09 00007f796e3bf730 00007f796e48579b 00007f796e485821 00007f796e4492b0 00007f796e4347f9 00007f796e4342a2 00007f7967b1a684
0809 15:53:04.951002 (+ 268us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:05.019508 (+ 68506us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:05.019536 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:05.032008 (+ 12472us) log.cc:842] Serialized 26 byte log entry
0809 15:53:05.375155 (+343147us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:53:05.375216 (+ 61us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:05.375306 (+ 90us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:05.430890 (+ 55584us) raft_consensus.cc:1671] finished
0809 15:53:05.430916 (+ 26us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:05.431185 (+ 269us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":42455,"raft.queue_time_us":57547,"raft.run_cpu_time_us":128,"raft.run_wall_time_us":7689,"spinlock_wait_cycles":26728704,"thread_start_us":301420,"threads_started":1,"wal-append.queue_time_us":316766}
I20260809 15:53:05.436609 11606 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:05.274147 14675 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:05.302834 14612 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:05.438131 14675 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:05.441515 14675 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:05.442714 14653 raft_consensus.cc:2726] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:05.129424 14641 raft_consensus.cc:2781] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:05.445923 14653 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:05.446321 14641 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:05.446956 14641 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:05.446620 14653 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:05.450598 14653 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:05.189221 14640 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:05.224140 14254 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:05.274719 14455 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:05.219529 14679 consensus_queue.cc:1035] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.385s
I20260809 15:53:05.175510 14402 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:05.415805 14684 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:05.451866 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:05.455408 14684 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:05.456620 11012 raft_consensus.cc:2379] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 4: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:05.424001 9552 auto_leader_rebalancer.cc:450] All tables' leader rebalancing finished this round
I20260809 15:53:05.309398 14658 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:05.243928 14540 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:05.437355 14697 raft_consensus.cc:2726] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:05.445199 11606 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:53:05.458683 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 44569.6us
I20260809 15:53:05.483035 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:53:05.483598 14561 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:05.486583 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 72602.2us
I20260809 15:53:05.458954 14654 raft_consensus.cc:2726] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223) with higher term. Message: Invalid argument: T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8.
I20260809 15:53:05.490052 14672 consensus_queue.cc:1035] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 15:53:05.490993 14676 raft_consensus.cc:2726] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260809 15:53:05.491910 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 46799.9us
I20260809 15:53:05.492149 14672 consensus_queue.cc:1046] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.001s
I20260809 15:53:05.493284 14672 raft_consensus.cc:3032] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 9 LEADER]: Stepping down as leader of term 9
I20260809 15:53:05.493641 14672 raft_consensus.cc:729] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:05.494596 10811 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:05.495487 14507 raft_consensus.cc:2726] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:05.497995 14540 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:05.498611 11010 raft_consensus.cc:1226] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 4. Current term is 5. Ops: []
I20260809 15:53:05.498687 10926 raft_consensus.cc:1226] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 5. Current term is 6. Ops: []
I20260809 15:53:05.500969 10811 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:05.377558 14280 raft_consensus.cc:2781] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
W20260809 15:53:05.512005 10927 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.566s user 0.013s sys 0.011s
W20260809 15:53:05.512786 10978 connection.cc:232] RPC call timeout handler was delayed by -1.5307580480002798s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1664s for a response
I20260809 15:53:05.524190 14280 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
W20260809 15:53:05.524358 10978 connection.cc:232] RPC call timeout handler was delayed by -1.4434948909911327s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.15800000000000003s for a response
W20260809 15:53:05.524852 10978 connection.cc:232] RPC call timeout handler was delayed by -1.3522870719898492s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.17490000000000003s for a response
W20260809 15:53:05.525408 10927 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.579s user 0.013s sys 0.011s
I20260809 15:53:05.516712 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:05.516687 11606 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:53:05.517951 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 507519us
I20260809 15:53:05.517590 11013 raft_consensus.cc:1261] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260809 15:53:05.525298 10978 connection.cc:232] RPC call timeout handler was delayed by -1.3272234960459173s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.2001s for a response
W20260809 15:53:05.526681 10978 connection.cc:232] RPC call timeout handler was delayed by -1.3091312780161388s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.26730000000000004s for a response
I20260809 15:53:05.526717 10893 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:05.519093 14672 consensus_queue.cc:260] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:05.527312 10978 connection.cc:232] RPC call timeout handler was delayed by -1.0449555779923685s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.16870000000000002s for a response
I20260809 15:53:05.458058 14684 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:53:05.517390 14048 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.526s user 0.007s sys 0.016s
I20260809 15:53:05.535405 11017 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 771) took 1395 ms. Trace:
I20260809 15:53:05.536190 11017 rpcz_store.cc:276] 0809 15:53:04.140317 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:04.814499 (+674182us) service_pool.cc:227] Handling call
0809 15:53:04.966769 (+152270us) spinlock_profiling.cc:243] Waited 8.44 ms on lock 0x7b5800027098. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f7966593851 00007f7966593702 00007f796f483b09 00007f796e3bf730 00007f796e48579b 00007f796e485821 00007f796e4492b0 00007f796e4347f9 00007f796e4342a2 00007f7967b1a684
0809 15:53:04.967040 (+ 271us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:05.069134 (+102094us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:05.069162 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:05.069206 (+ 44us) raft_consensus.cc:1631] Marking committed up to 8
0809 15:53:05.069237 (+ 31us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:05.535181 (+465944us) spinlock_profiling.cc:243] Waited 102 ms on lock 0x7b540040a454. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:05.535230 (+ 49us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:05.535351 (+ 121us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":253703808}
I20260809 15:53:05.514370 14405 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:05.537827 11606 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:05.490295 14527 consensus_queue.cc:1035] T b6ba8250f4ef43bfbf8f517e86a131a1 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 6, Time since last communication: 0.039s
I20260809 15:53:05.538005 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:05.516075 14619 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 12 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:05.418434 14503 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:05.539049 10893 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:53:05.531944 14675 raft_consensus.cc:2726] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:53:05.529428 14247 raft_consensus.cc:2726] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260809 15:53:05.530839 14621 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.636s user 0.006s sys 0.021s
I20260809 15:53:05.524786 14280 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:53:05.543699 11606 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:53:05.544158 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:53:05.536136 14048 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.544s user 0.007s sys 0.016s
I20260809 15:53:05.465770 14638 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:05.548456 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:05.565980 10893 leader_election.cc:398] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Vote denied by peer a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615) with higher term. Message: Invalid argument: T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8.
I20260809 15:53:05.567682 11606 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:05.550541 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:05.538489 14254 raft_consensus.cc:2726] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:53:05.574529 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:05.575656 14638 raft_consensus.cc:2726] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 15:53:05.571502 11015 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:53:05.576475 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:05.577082 10891 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:05.578146 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:05.572156 10923 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:05.578359 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:05.579281 10844 raft_consensus.cc:2396] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 7 because replica has last-logged OpId of term: 6 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 2.
I20260809 15:53:05.574345 14648 raft_consensus.cc:2781] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:53:05.582337 14648 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:05.582783 14648 raft_consensus.cc:3037] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:05.584450 11633 raft_consensus.cc:1261] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 7. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260809 15:53:05.586206 14675 raft_consensus.cc:2781] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election won for term 7
W20260809 15:53:05.573173 14621 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.692s user 0.007s sys 0.023s
I20260809 15:53:05.574441 10837 raft_consensus.cc:1261] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 8 index: 9. (term mismatch)
I20260809 15:53:05.589258 10837 pending_rounds.cc:76] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536: Aborting all ops after (but not including) 8
I20260809 15:53:05.589468 10837 pending_rounds.cc:97] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536: Aborting uncommitted NO_OP operation due to leader change: 9.9
I20260809 15:53:05.589744 10837 raft_consensus.cc:2866] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:53:05.591293 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:05.591926 11013 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 12.
I20260809 15:53:05.592257 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:05.592792 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:05.594848 11633 raft_consensus.cc:1261] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260809 15:53:05.594529 11015 raft_consensus.cc:2379] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 8: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:05.533550 14507 consensus_queue.cc:1035] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.016s
I20260809 15:53:05.597954 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:05.598225 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:05.602391 10923 raft_consensus.cc:2454] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 8.
I20260809 15:53:05.603928 10837 raft_consensus.cc:2379] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 7: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:53:05.605453 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:05.605697 10839 raft_consensus.cc:1261] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260809 15:53:05.606570 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:05.607900 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:05.608605 13973 raft_consensus.cc:2726] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:53:05.609289 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:05.609450 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:05.610270 10838 raft_consensus.cc:2379] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 6: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:05.609979 10848 raft_consensus.cc:2379] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 6: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:53:05.611564 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:05.612757 10978 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:05.609310 14641 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:05.613827 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 63885.5us
I20260809 15:53:05.614481 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:05.615144 10978 leader_election.cc:398] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Vote denied by peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685) with higher term. Message: Invalid argument: T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5.
I20260809 15:53:05.615447 10978 leader_election.cc:401] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Cancelling election due to peer responding with higher term
I20260809 15:53:05.615674 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:05.616266 11641 raft_consensus.cc:2454] T e6e09d263f96440a99c99d5a5c6ea477 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 10.
I20260809 15:53:05.616458 11015 raft_consensus.cc:1226] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4. Ops: []
I20260809 15:53:05.617837 14701 raft_consensus.cc:2726] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 15:53:05.618062 11634 raft_consensus.cc:1261] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260809 15:53:05.618949 10978 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:05.619854 14436 raft_consensus.cc:2726] T b6d399a7394c4a59b90a308aedfdf899 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 15:53:05.621702 14561 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:05.622350 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:05.624995 14503 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:05.625519 14654 raft_consensus.cc:2781] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:05.625880 14654 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:05.626226 14654 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:05.626474 14672 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Advancing to term 10
I20260809 15:53:05.619007 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:05.633015 11640 raft_consensus.cc:2354] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for earlier term 7. Current term is 8.
I20260809 15:53:05.634194 11642 raft_consensus.cc:2379] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 9: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:05.610318 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:05.545204 14653 raft_consensus.cc:2726] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:05.638285 11641 raft_consensus.cc:1261] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260809 15:53:05.640326 10929 raft_consensus.cc:1226] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260809 15:53:05.542752 14540 consensus_queue.cc:1035] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.063s
I20260809 15:53:05.549332 14619 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 12 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:05.645292 14619 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 13 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:05.545760 14405 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:05.648064 14405 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:05.648890 14254 consensus_queue.cc:1046] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 4.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:53:05.650336 11011 raft_consensus.cc:1226] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:05.650913 11011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2295) took 1647 ms. Trace:
I20260809 15:53:05.651216 11011 rpcz_store.cc:276] 0809 15:53:04.003349 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:04.808055 (+804706us) service_pool.cc:227] Handling call
0809 15:53:05.649920 (+841865us) spinlock_profiling.cc:243] Waited 14 ms on lock 0x7b5800027098. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f7966593851 00007f7966593702 00007f796f483b09 00007f796e3bf730 00007f796e48579b 00007f796e485821 00007f796e4492b0 00007f796e4347f9 00007f796e4342a2 00007f7967b1a684
0809 15:53:05.650128 (+ 208us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:05.650695 (+ 567us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:05.650861 (+ 166us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":32233216}
I20260809 15:53:05.651863 14685 raft_consensus.cc:3032] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:53:05.652252 14685 raft_consensus.cc:729] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:53:05.652334 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:05.636852 14648 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:05.656921 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:05.657500 11637 raft_consensus.cc:3037] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
W20260809 15:53:05.667030 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 127760us
I20260809 15:53:05.671845 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:05.672716 11639 raft_consensus.cc:1226] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: []
I20260809 15:53:05.671846 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:53:05.673290 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 54423.6us
I20260809 15:53:05.676779 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:05.673960 10978 leader_election.cc:302] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:05.681303 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:05.681881 11642 raft_consensus.cc:2454] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:53:05.682230 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:05.682907 11635 raft_consensus.cc:3037] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:53:05.684058 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:05.684564 11606 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:05.684578 11642 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Advancing to term 10
I20260809 15:53:05.589535 14625 consensus_queue.cc:260] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:05.689029 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:05.689646 10843 raft_consensus.cc:1226] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 5. Current term is 6. Ops: []
I20260809 15:53:05.696866 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:05.696899 10838 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 12.
I20260809 15:53:05.615281 14641 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:05.690750 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:05.687472 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:05.695843 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:05.695899 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:05.696307 10848 raft_consensus.cc:1261] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 6. Preceding OpId from leader: term: 4 index: 6. (term mismatch)
I20260809 15:53:05.635241 14675 raft_consensus.cc:686] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:05.653005 14685 consensus_queue.cc:260] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:05.877789 10930 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54224 (request call id 2130) took 1136 ms. Trace:
I20260809 15:53:05.853204 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:05.833884 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:05.891073 10927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 2138) took 1125 ms. Trace:
I20260809 15:53:05.913822 10930 rpcz_store.cc:276] 0809 15:53:04.741349 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:04.741536 (+ 187us) service_pool.cc:227] Handling call
0809 15:53:05.877605 (+1136069us) spinlock_profiling.cc:243] Waited 204 ms on lock 0x7b5400372654. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:05.877727 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":469955968}
I20260809 15:53:05.886482 11009 raft_consensus.cc:1226] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 3. Current term is 4. Ops: [3.5-3.5]
I20260809 15:53:05.811141 11642 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 10.
I20260809 15:53:05.782346 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:05.757306 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
W20260809 15:53:05.761034 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 58291.8us
I20260809 15:53:05.915294 10848 pending_rounds.cc:76] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536: Aborting all ops after (but not including) 5
I20260809 15:53:05.756539 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:05.811833 11635 raft_consensus.cc:2454] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:53:05.697511 10840 raft_consensus.cc:2454] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:53:05.759079 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:53:05.869275 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 195258us
I20260809 15:53:05.759351 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:05.926996 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:05.927256 10978 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
W20260809 15:53:05.782823 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 109293us
I20260809 15:53:05.928618 11634 raft_consensus.cc:3037] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 3 FOLLOWER]: Advancing to term 4
I20260809 15:53:05.916771 10839 raft_consensus.cc:2454] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 7.
I20260809 15:53:05.916429 11606 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:05.915695 11640 raft_consensus.cc:2454] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:53:05.756256 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:05.933980 11018 raft_consensus.cc:1226] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 6. Current term is 7. Ops: []
W20260809 15:53:05.839489 14702 log.cc:924] Time spent T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.209s user 0.002s sys 0.001s
I20260809 15:53:05.935395 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:05.935349 11009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2499) took 1024 ms. Trace:
W20260809 15:53:05.815646 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 126876us
I20260809 15:53:05.820179 11636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 1010) took 2803 ms. Trace:
I20260809 15:53:05.902365 14487 raft_consensus.cc:3032] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 15:53:05.948292 10811 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
W20260809 15:53:05.816761 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 179487us
I20260809 15:53:05.758749 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:05.948724 10978 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:53:05.942517 10848 pending_rounds.cc:97] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536: Aborting uncommitted NO_OP operation due to leader change: 3.6
I20260809 15:53:05.852056 11637 raft_consensus.cc:2454] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:53:05.889812 14648 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:05.699664 10929 raft_consensus.cc:1226] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 8. Current term is 9. Ops: []
I20260809 15:53:05.848204 10924 pending_rounds.cc:76] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3: Aborting all ops after (but not including) 5
I20260809 15:53:05.963630 10924 pending_rounds.cc:97] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260809 15:53:05.963956 10924 raft_consensus.cc:2866] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:53:05.927651 11642 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 12.
I20260809 15:53:05.943130 11636 rpcz_store.cc:276] 0809 15:53:03.016381 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:03.545300 (+528919us) service_pool.cc:227] Handling call
0809 15:53:03.918035 (+372735us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:04.838625 (+920590us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:05.819910 (+981285us) spinlock_profiling.cc:243] Waited 868 ms on lock 0x7b540066c454. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:05.820110 (+ 200us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1995766016}
I20260809 15:53:05.946568 11017 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 12.
I20260809 15:53:05.936275 11641 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:53:05.952270 10848 raft_consensus.cc:2866] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:53:05.936273 11009 rpcz_store.cc:276] 0809 15:53:04.911274 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:05.582041 (+670767us) service_pool.cc:227] Handling call
0809 15:53:05.886226 (+304185us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:05.918309 (+ 32083us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:05.935193 (+ 16884us) spinlock_profiling.cc:243] Waited 304 ms on lock 0x7b54002db250. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:05.935300 (+ 107us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":698699520}
W20260809 15:53:05.702229 14689 log.cc:924] Time spent T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.327s user 0.002s sys 0.000s
W20260809 15:53:05.975721 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 59372.3us
I20260809 15:53:05.807215 14492 consensus_queue.cc:1046] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.174s
I20260809 15:53:05.703329 14684 consensus_queue.cc:1046] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.278s
I20260809 15:53:05.964388 14405 raft_consensus.cc:2781] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:05.992238 14405 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:05.992780 14405 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
W20260809 15:53:05.994750 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 46097.8us
I20260809 15:53:05.995788 10978 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:53:05.996811 14407 raft_consensus.cc:2781] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:05.996994 14254 raft_consensus.cc:2726] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:53:05.997226 14407 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:05.997371 14612 raft_consensus.cc:2781] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:53:05.997680 14407 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:05.998008 14612 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:05.998972 14612 raft_consensus.cc:3037] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:06.001948 14394 raft_consensus.cc:482] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:06.002494 14394 raft_consensus.cc:504] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:06.004832 14394 leader_election.cc:288] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:06.005507 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:53:06.006428 10893 connection.cc:232] RPC call timeout handler was delayed by -1.1978200200246647s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1504s for a response
I20260809 15:53:05.847170 14658 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:06.031613 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 83557.6us
I20260809 15:53:06.032806 10811 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:05.915995 10927 rpcz_store.cc:276] 0809 15:53:04.765320 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:04.923429 (+158109us) service_pool.cc:227] Handling call
0809 15:53:04.930424 (+ 6995us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:04.931310 (+ 886us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:04.931342 (+ 32us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:04.937186 (+ 5844us) log.cc:842] Serialized 26 byte log entry
0809 15:53:05.525970 (+588784us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:53:05.542301 (+ 16331us) log.cc:842] Serialized 14 byte log entry
0809 15:53:05.542698 (+ 397us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:05.542767 (+ 69us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:05.836183 (+293416us) spinlock_profiling.cc:243] Waited 2.39 ms on lock 0x7b1800449980. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f79665cb638 00007f79665ceb11 00007f79665ceac9 00007f79665cea59 00007f79665cea21 00007f79665cdd1d 00007f796f82f204 00007f796f82f039 00007f79665addc6
0809 15:53:05.890685 (+ 54502us) raft_consensus.cc:1671] finished
0809 15:53:05.890821 (+ 136us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:05.890994 (+ 173us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":691,"prepare.queue_time_us":16181,"prepare.run_cpu_time_us":37,"prepare.run_wall_time_us":37,"raft.queue_time_us":966,"raft.run_cpu_time_us":244,"raft.run_wall_time_us":6068,"spinlock_wait_cycles":5494144,"thread_start_us":579798,"threads_started":1,"wal-append.queue_time_us":811555}
I20260809 15:53:06.035449 11606 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:06.038327 10893 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:53:05.892624 14654 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:05.945971 14487 raft_consensus.cc:729] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:53:06.020349 10978 leader_election.cc:302] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:06.020628 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:06.055747 10929 raft_consensus.cc:2379] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 7: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:06.022723 14697 raft_consensus.cc:2726] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:53:06.024405 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:06.009867 14685 raft_consensus.cc:3037] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:53:06.025302 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:06.025411 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:06.025497 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:06.026178 14666 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:06.029415 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:05.962900 14280 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:06.030244 14537 consensus_queue.cc:1046] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.000s
W20260809 15:53:06.058929 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 51053.4us
I20260809 15:53:06.078940 11606 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:06.024794 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:06.079901 10930 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
W20260809 15:53:06.084730 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 52003.2us
I20260809 15:53:06.088271 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:05.759785 14540 raft_consensus.cc:2726] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:53:06.064507 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:06.072945 11009 raft_consensus.cc:1261] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 15:53:06.063230 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:06.076073 10920 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:06.077102 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:06.052397 14394 raft_consensus.cc:2781] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:53:06.095518 11606 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:06.059357 14658 raft_consensus.cc:2726] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:53:06.061239 14402 raft_consensus.cc:2781] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:53:06.103363 10929 raft_consensus.cc:1226] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 9. Current term is 10. Ops: []
I20260809 15:53:06.103912 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:06.060706 14521 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:06.110945 11606 leader_election.cc:302] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:06.078310 14701 raft_consensus.cc:2726] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 4. Reason: Vote denied by peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685) with higher term. Message: Invalid argument: T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5.
I20260809 15:53:06.108659 10893 leader_election.cc:302] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:06.111603 14643 raft_consensus.cc:2726] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260809 15:53:06.116062 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:06.098583 14261 raft_consensus.cc:2781] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 15:53:06.046268 14407 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:06.126389 14407 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:06.116302 14521 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:06.128166 14521 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:06.117334 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:06.128822 10927 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
I20260809 15:53:06.097826 14648 raft_consensus.cc:2726] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 15:53:06.130122 10978 leader_election.cc:398] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Vote denied by peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685) with higher term. Message: Invalid argument: T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 5. Current term is 6.
I20260809 15:53:06.122640 10931 raft_consensus.cc:2454] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:53:06.130439 10978 leader_election.cc:401] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Cancelling election due to peer responding with higher term
I20260809 15:53:06.131539 14652 consensus_queue.cc:1035] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 7, Time since last communication: 0.001s
I20260809 15:53:06.132083 10978 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:06.132325 10893 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:06.122849 10920 raft_consensus.cc:2454] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:53:06.133283 10978 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:06.124104 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:06.124540 10929 raft_consensus.cc:1226] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 5. Current term is 6. Ops: []
I20260809 15:53:06.103919 14641 raft_consensus.cc:2726] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:06.140523 10926 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 12.
I20260809 15:53:06.144870 10930 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 8.
I20260809 15:53:06.099233 14394 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:06.150324 14642 raft_consensus.cc:3032] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:53:06.150708 14652 raft_consensus.cc:2726] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:06.148041 14261 raft_consensus.cc:686] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [term 8 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:06.151548 14487 consensus_queue.cc:260] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:06.153258 14654 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:06.159330 10930 raft_consensus.cc:1226] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 8. Current term is 9. Ops: []
I20260809 15:53:06.161499 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:06.164662 11009 raft_consensus.cc:1226] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:06.134426 14402 raft_consensus.cc:686] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [term 6 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:06.087675 14666 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:06.166241 11018 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:53:06.156297 14394 raft_consensus.cc:3037] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:06.121917 13806 consensus_queue.cc:1035] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260809 15:53:06.169471 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:06.169689 10891 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:06.157843 14658 raft_consensus.cc:2726] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:06.156236 14653 raft_consensus.cc:2726] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority
I20260809 15:53:06.172261 11016 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:06.159401 14540 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 12 FOLLOWER]: Leader election won for term 12
I20260809 15:53:06.156097 14642 raft_consensus.cc:729] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:53:06.105592 14675 consensus_queue.cc:237] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:06.148286 14521 consensus_queue.cc:1035] T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.024s
I20260809 15:53:06.154992 14473 consensus_queue.cc:1046] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.024s
I20260809 15:53:06.154704 14700 consensus_queue.cc:1035] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260809 15:53:06.181056 10921 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Advancing to term 10
I20260809 15:53:06.171715 14487 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:53:06.194595 10978 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:53:06.196247 14709 log.cc:924] Time spent T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.088s user 0.000s sys 0.000s
I20260809 15:53:06.198448 14666 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:06.199342 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:06.199834 14473 raft_consensus.cc:3032] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:53:06.200273 14473 raft_consensus.cc:729] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:06.200351 11013 raft_consensus.cc:2379] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 10: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:06.201117 14473 consensus_queue.cc:260] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:06.206269 14492 consensus_queue.cc:1046] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.000s
I20260809 15:53:06.207201 14164 raft_consensus.cc:3032] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 15:53:06.207552 14164 raft_consensus.cc:729] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:06.207883 14642 consensus_queue.cc:260] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:06.208323 14164 consensus_queue.cc:260] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:06.209136 14642 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:06.209498 14164 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:06.213095 14473 raft_consensus.cc:3037] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:06.224838 11634 raft_consensus.cc:2454] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:53:06.228672 11016 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 8.
I20260809 15:53:06.229341 10921 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 10.
I20260809 15:53:06.230914 11011 raft_consensus.cc:1226] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:06.213862 14625 raft_consensus.cc:3037] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 3 FOLLOWER]: Advancing to term 4
W20260809 15:53:06.235551 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 52719us
I20260809 15:53:06.179767 14679 raft_consensus.cc:2781] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:06.188457 14540 raft_consensus.cc:686] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 12 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
W20260809 15:53:06.237174 14634 log.cc:924] Time spent T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.054s user 0.000s sys 0.000s
I20260809 15:53:06.240572 14701 raft_consensus.cc:2741] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election decision vote started in defunct term 7: won
W20260809 15:53:06.332935 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 79551.3us
I20260809 15:53:06.263831 14679 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:06.337970 10891 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:06.270856 14521 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:06.270784 14280 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:06.353590 10978 leader_election.cc:398] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 pre-election: Vote denied by peer 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783) with higher term. Message: Invalid argument: T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5.
I20260809 15:53:06.355278 14540 consensus_queue.cc:237] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 10.10, Last appended by leader: 10, Current term: 12, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:06.343782 14679 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:06.270682 14247 consensus_queue.cc:1046] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.077s
I20260809 15:53:06.412962 14701 consensus_queue.cc:1046] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 3.2, Next index: 3, Last known committed idx: 2, Time since last communication: 0.001s
I20260809 15:53:06.346848 14621 raft_consensus.cc:2726] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:06.422299 14521 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:06.424919 14521 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:06.272830 14261 consensus_queue.cc:237] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 5.4, Last appended by leader: 4, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:06.347882 14652 raft_consensus.cc:2781] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 15:53:06.434681 14652 raft_consensus.cc:686] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 8 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:06.435806 14652 consensus_queue.cc:237] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 6.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:06.447595 14612 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:06.463594 14655 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:06.464226 14655 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:06.451735 14405 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:06.250576 14402 consensus_queue.cc:237] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:06.460758 14701 raft_consensus.cc:3032] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:53:06.488118 14701 raft_consensus.cc:729] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:06.489125 14701 consensus_queue.cc:260] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:53:06.477329 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 123912us
I20260809 15:53:06.389299 14394 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:06.505486 11014 raft_consensus.cc:1226] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:06.508072 14655 leader_election.cc:288] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:06.515967 11013 raft_consensus.cc:1226] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 5. Current term is 6. Ops: []
I20260809 15:53:06.504815 14405 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:06.518777 14527 raft_consensus.cc:3032] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 15:53:06.563598 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:06.558070 14527 raft_consensus.cc:729] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:06.593582 10920 raft_consensus.cc:1226] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 5. Current term is 6. Ops: []
I20260809 15:53:06.547161 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
W20260809 15:53:06.602510 11017 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.527s user 0.006s sys 0.030s
W20260809 15:53:06.602965 11017 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.527s user 0.006s sys 0.030s
W20260809 15:53:06.605336 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 473040us
I20260809 15:53:06.606285 10893 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:53:06.607367 10893 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:53:06.608489 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 498366us
I20260809 15:53:06.609362 11606 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:06.610373 14728 raft_consensus.cc:2726] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260809 15:53:06.613960 14503 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.550s user 0.019s sys 0.018s
W20260809 15:53:06.614487 14503 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.551s user 0.019s sys 0.018s
I20260809 15:53:06.623143 14729 raft_consensus.cc:2781] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:53:06.623754 14729 raft_consensus.cc:686] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 6 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:06.625311 14655 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260809 15:53:06.625797 14655 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:06.626132 14655 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 12 FOLLOWER]: Advancing to term 13
I20260809 15:53:06.630466 14729 consensus_queue.cc:237] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:06.640281 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
W20260809 15:53:06.646663 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 46882.7us
I20260809 15:53:06.647778 11606 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:53:06.652527 11015 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.501s user 0.126s sys 0.139s
W20260809 15:53:06.652981 11015 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.502s user 0.126s sys 0.139s
I20260809 15:53:06.657217 11639 raft_consensus.cc:1261] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260809 15:53:06.657631 11639 pending_rounds.cc:76] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9: Aborting all ops after (but not including) 5
I20260809 15:53:06.657857 11639 pending_rounds.cc:97] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260809 15:53:06.658227 11639 raft_consensus.cc:2866] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:53:06.660106 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:06.660719 11641 raft_consensus.cc:2454] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:53:06.666437 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:06.671056 10978 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:06.672518 10978 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:06.656292 14407 raft_consensus.cc:2726] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election lost for term 5. Reason: Vote denied by peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685) with higher term. Message: Invalid argument: T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 5. Current term is 6.
I20260809 15:53:06.681501 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:06.683338 10840 raft_consensus.cc:1261] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
I20260809 15:53:06.656751 14730 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:06.686653 14730 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:06.667536 14701 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:06.699275 11639 raft_consensus.cc:1226] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:06.705363 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:06.706048 11642 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:06.700836 10893 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:06.699846 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:06.715996 11642 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 8.
I20260809 15:53:06.647801 14405 consensus_queue.cc:1035] T fe21e0688ade405bb4590b9c256bd696 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.029s
I20260809 15:53:06.673926 14658 raft_consensus.cc:2726] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 15:53:06.674008 14733 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:53:06.722031 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 53364.4us
I20260809 15:53:06.723219 10893 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:06.689245 14730 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:06.684334 14254 raft_consensus.cc:2726] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260809 15:53:06.675372 14455 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.613s user 0.020s sys 0.022s
I20260809 15:53:06.659657 14697 consensus_queue.cc:1046] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.044s
I20260809 15:53:06.663761 14674 consensus_queue.cc:1035] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.054s
W20260809 15:53:06.747363 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 95778.4us
I20260809 15:53:06.722117 14654 raft_consensus.cc:2726] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:53:06.745291 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:06.758983 11639 raft_consensus.cc:2379] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 10: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:06.759019 14348 consensus_queue.cc:1046] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260809 15:53:06.759927 14625 consensus_meta.cc:287] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.510s user 0.009s sys 0.000s
I20260809 15:53:06.760596 10922 raft_consensus.cc:2454] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:53:06.728875 14247 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:06.717634 14492 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:53:06.773468 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:06.773726 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:06.774549 10842 raft_consensus.cc:2379] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 10: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:06.745898 14658 raft_consensus.cc:2726] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260809 15:53:06.751152 14473 consensus_meta.cc:287] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.538s user 0.011s sys 0.000s
I20260809 15:53:06.776294 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:06.786170 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:53:06.749811 14455 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.683s user 0.020s sys 0.022s
I20260809 15:53:06.787012 10847 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 10.
W20260809 15:53:06.789829 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 72685.6us
I20260809 15:53:06.790762 11011 raft_consensus.cc:1261] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 2. Preceding OpId from leader: term: 7 index: 3. (index mismatch)
W20260809 15:53:06.776959 14487 consensus_meta.cc:287] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.542s user 0.011s sys 0.000s
I20260809 15:53:06.795176 11635 raft_consensus.cc:2379] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 10: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:53:06.798663 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:06.766726 14700 raft_consensus.cc:2726] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 15:53:06.798808 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:06.801446 14394 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:06.802058 14744 log.cc:924] Time spent T 7c8eaf50bcb3457bb4b1e30406cc349f P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.101s user 0.000s sys 0.000s
I20260809 15:53:06.775321 14247 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:06.788146 14612 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:06.813062 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 140638us
I20260809 15:53:06.816970 10978 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:06.817386 10893 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:06.817975 10978 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:53:06.822526 9552 auto_leader_rebalancer.cc:140] leader rebalance for table test-workload
I20260809 15:53:06.822932 10840 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
I20260809 15:53:06.821740 14540 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:06.823704 14540 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:06.825971 14540 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:06.828166 11012 raft_consensus.cc:1226] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:06.831305 10893 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:06.774600 14402 consensus_queue.cc:1046] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.019s
I20260809 15:53:06.776935 14492 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:06.744627 14733 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:06.771143 14164 consensus_queue.cc:1046] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260809 15:53:06.838937 10811 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:53:06.859853 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 60670.8us
I20260809 15:53:06.833333 14640 raft_consensus.cc:3032] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:53:06.834661 14405 raft_consensus.cc:2726] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
W20260809 15:53:06.857645 14737 log.cc:924] Time spent T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.225s user 0.001s sys 0.000s
W20260809 15:53:06.884678 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 53460.5us
I20260809 15:53:06.856668 14612 raft_consensus.cc:2781] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:53:06.895097 10924 raft_consensus.cc:1226] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:06.891033 10893 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:06.898882 10922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52758 (request call id 891) took 1216 ms. Trace:
I20260809 15:53:06.899268 10922 rpcz_store.cc:276] 0809 15:53:05.682403 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:05.686593 (+ 4190us) service_pool.cc:227] Handling call
0809 15:53:06.898737 (+1212144us) spinlock_profiling.cc:243] Waited 834 ms on lock 0x7b540018ae54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:06.898829 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1917513088}
I20260809 15:53:06.901352 10923 raft_consensus.cc:1261] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 15:53:06.861142 14503 raft_consensus.cc:2726] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority
I20260809 15:53:06.907658 10978 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
W20260809 15:53:06.953567 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 45993us
W20260809 15:53:06.953920 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 46931.1us
I20260809 15:53:06.954648 10893 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:06.836247 14048 consensus_queue.cc:1046] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 9.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.058s
I20260809 15:53:06.877894 14640 raft_consensus.cc:729] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:53:06.856849 14625 consensus_queue.cc:1035] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.067s
I20260809 15:53:06.855393 14621 consensus_queue.cc:1035] T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.047s
I20260809 15:53:06.972457 10978 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:06.893851 14612 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:06.889369 14730 raft_consensus.cc:2726] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:53:06.889238 14666 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:53:06.968129 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 211756us
I20260809 15:53:06.959401 14700 raft_consensus.cc:2726] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:53:06.988422 14455 raft_consensus.cc:2726] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:06.993637 10978 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:07.017175 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:07.018302 10837 raft_consensus.cc:2379] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 5: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:53:07.062634 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:06.962656 14749 raft_consensus.cc:2726] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
W20260809 15:53:07.001719 14752 log.cc:924] Time spent T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.167s user 0.000s sys 0.000s
I20260809 15:53:06.998215 14164 raft_consensus.cc:2781] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:07.039528 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:07.080379 14048 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:07.022112 10893 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:07.099664 10841 raft_consensus.cc:1226] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: []
I20260809 15:53:06.994652 14674 raft_consensus.cc:2781] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election won for term 6
W20260809 15:53:07.101562 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 79583us
I20260809 15:53:07.050910 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:07.109050 10893 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:06.995016 14612 raft_consensus.cc:3037] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:07.110292 10893 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:07.108963 10847 raft_consensus.cc:2454] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
W20260809 15:53:07.041703 14721 log.cc:924] Time spent T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.070s user 0.000s sys 0.000s
I20260809 15:53:07.104941 14164 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:07.120149 10893 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:07.084391 14771 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:07.084511 14402 raft_consensus.cc:3032] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 9 LEADER]: Stepping down as leader of term 9
W20260809 15:53:07.124226 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 285370us
I20260809 15:53:07.110333 14749 raft_consensus.cc:2781] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election won for term 8
I20260809 15:53:07.142275 14775 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:53:07.142890 14679 consensus_meta.cc:287] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.749s user 0.010s sys 0.000s
I20260809 15:53:07.142782 14775 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:07.143481 14679 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:07.144965 14775 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:07.146199 14729 raft_consensus.cc:2726] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 15:53:07.147197 14402 raft_consensus.cc:729] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 9 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:07.148384 14402 consensus_queue.cc:260] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 9.5, Last appended by leader: 5, Current term: 9, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:07.150182 14733 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:07.100514 14673 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:53:07.150701 14751 log.cc:924] Time spent T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.099s user 0.000s sys 0.000s
I20260809 15:53:07.105870 14247 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:07.111011 14730 raft_consensus.cc:2726] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:07.131599 10893 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:53:06.998782 14684 consensus_queue.cc:1035] T 02f25d93293a40729fd751a19ab9adeb P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.120s
I20260809 15:53:07.137279 10811 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:53:07.138418 14655 consensus_meta.cc:287] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.512s user 0.009s sys 0.002s
I20260809 15:53:07.122622 14700 raft_consensus.cc:2726] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 15:53:06.979326 14527 consensus_queue.cc:260] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:07.123204 14621 raft_consensus.cc:2726] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260809 15:53:07.134276 14164 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:07.166657 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:07.023644 14666 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:07.165277 10893 leader_election.cc:302] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:07.174233 11012 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:07.063444 14652 consensus_queue.cc:1035] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.092s
I20260809 15:53:07.176337 10811 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 13 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:07.155298 14679 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:07.112573 14048 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:07.123288 14674 raft_consensus.cc:686] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:07.188081 14048 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:07.188535 14750 log.cc:924] Time spent T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.326s user 0.001s sys 0.000s
I20260809 15:53:07.178946 14621 raft_consensus.cc:2726] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:07.165297 14655 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:07.155426 14771 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:07.193112 14766 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 12 FOLLOWER]: Leader pre-election won for term 13
I20260809 15:53:07.171629 14463 consensus_queue.cc:1046] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 7.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.022s
I20260809 15:53:07.213333 14733 consensus_queue.cc:1046] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260809 15:53:07.113924 14640 consensus_queue.cc:260] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:07.173215 14492 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:07.203130 14679 raft_consensus.cc:482] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:07.171689 14675 raft_consensus.cc:2781] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 15:53:07.216166 14527 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:07.217029 14771 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:07.194149 14247 consensus_queue.cc:1046] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.011s
W20260809 15:53:07.221580 14701 consensus_meta.cc:287] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.504s user 0.007s sys 0.000s
I20260809 15:53:07.179478 14769 raft_consensus.cc:2781] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:53:07.246389 14679 raft_consensus.cc:504] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:07.155907 14673 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:07.251930 14673 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:07.221939 14048 consensus_queue.cc:1046] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 8.5, Next index: 6, Last known committed idx: 4, Time since last communication: 0.000s
I20260809 15:53:07.252095 14679 leader_election.cc:288] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:07.253036 14048 raft_consensus.cc:3032] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 15:53:07.253425 14048 raft_consensus.cc:729] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:07.224937 14729 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:07.226173 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:07.254760 10924 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 10.
I20260809 15:53:07.227725 10920 raft_consensus.cc:1226] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 7. Current term is 8. Ops: [7.5-7.5]
I20260809 15:53:07.228662 14666 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:07.204672 14700 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:07.164371 14775 consensus_queue.cc:1046] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 5.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.031s
I20260809 15:53:07.228906 10893 leader_election.cc:302] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:07.226200 14503 raft_consensus.cc:482] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:07.232930 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:07.213348 14655 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 13 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:07.234258 14766 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 12 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:07.225533 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:07.236407 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:07.267549 10923 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
W20260809 15:53:07.234414 14743 log.cc:924] Time spent T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.241s user 0.001s sys 0.000s
I20260809 15:53:07.250197 14247 raft_consensus.cc:3032] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:53:07.256563 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:07.264401 14612 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:07.265465 11012 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 8.
I20260809 15:53:07.238343 14675 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:07.254447 14640 raft_consensus.cc:3037] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:07.280174 10891 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:53:07.273293 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 44435.6us
I20260809 15:53:07.250345 14769 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:53:07.277848 14749 raft_consensus.cc:686] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [term 8 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:07.259903 14729 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:07.283356 14769 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:07.283977 14749 consensus_queue.cc:237] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:07.285346 14729 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:07.254240 14048 consensus_queue.cc:260] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:07.287179 14048 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:53:07.291375 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:07.292191 10838 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 10.
I20260809 15:53:07.293473 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:07.295591 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:07.303774 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:53:07.308979 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 287809us
I20260809 15:53:07.310163 14674 consensus_queue.cc:237] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:53:07.311605 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 228747us
I20260809 15:53:07.311722 11606 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:07.314397 14763 raft_consensus.cc:2726] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:07.318982 11606 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:07.322731 14487 raft_consensus.cc:2741] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election decision vote started in defunct term 3: won
I20260809 15:53:07.323235 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:07.323899 11633 raft_consensus.cc:3037] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:07.326555 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:07.328517 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:07.329159 11642 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 9.
I20260809 15:53:07.330613 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:07.332435 10920 raft_consensus.cc:1226] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 5. Current term is 6. Ops: []
I20260809 15:53:07.332337 14164 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:07.334429 11637 raft_consensus.cc:1261] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260809 15:53:07.336789 11642 raft_consensus.cc:1261] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260809 15:53:07.339628 14706 raft_consensus.cc:3032] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 15:53:07.340091 14706 raft_consensus.cc:729] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:07.340982 14706 consensus_queue.cc:260] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:07.351534 10923 raft_consensus.cc:2454] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:53:07.352727 11633 raft_consensus.cc:2454] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:53:07.359525 10923 raft_consensus.cc:1226] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260809 15:53:07.273144 14700 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:07.379563 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:07.380577 11014 raft_consensus.cc:2379] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 8: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
W20260809 15:53:07.382030 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 101955us
W20260809 15:53:07.371850 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 83342.5us
I20260809 15:53:07.389313 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:53:07.359520 11606 connection.cc:232] RPC call timeout handler was delayed by -1.2077506199711934s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.1574s for a response
I20260809 15:53:07.266196 14701 raft_consensus.cc:3032] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 LEADER]: Stepping down as leader of term 7
I20260809 15:53:07.273856 14766 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 12 FOLLOWER]: Advancing to term 13
I20260809 15:53:07.419691 10923 pending_rounds.cc:76] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3: Aborting all ops after (but not including) 4
W20260809 15:53:07.411891 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 304777us
I20260809 15:53:07.417990 10839 raft_consensus.cc:1261] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260809 15:53:07.419993 10923 pending_rounds.cc:97] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3: Aborting uncommitted NO_OP operation due to leader change: 3.5
I20260809 15:53:07.273620 14247 raft_consensus.cc:729] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:07.425740 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:07.344023 14675 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Advancing to term 10
I20260809 15:53:07.432627 10848 raft_consensus.cc:2454] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 8.
I20260809 15:53:07.434012 10848 raft_consensus.cc:1226] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 4. Current term is 5. Ops: []
I20260809 15:53:07.436533 11640 raft_consensus.cc:1226] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 5. Current term is 6. Ops: [3.6-5.7]
I20260809 15:53:07.420459 10923 raft_consensus.cc:2866] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:53:07.423103 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:07.438397 11633 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
I20260809 15:53:07.423337 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:07.424149 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:07.278234 14655 raft_consensus.cc:2781] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:53:07.426452 10843 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:07.440636 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:07.426949 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:07.427539 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:07.427769 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:07.442798 10842 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 9.
W20260809 15:53:07.442947 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 235179us
I20260809 15:53:07.422485 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:07.423108 10978 leader_election.cc:302] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:07.431561 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:07.443562 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:07.278632 14503 raft_consensus.cc:504] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:07.446444 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:07.446698 11606 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:53:07.451923 10811 leader_election.cc:334] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.647s (SENT)
I20260809 15:53:07.452486 10811 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:07.462778 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:07.464112 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:07.439868 11642 raft_consensus.cc:2454] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:53:07.464718 10978 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:07.469044 10838 raft_consensus.cc:1261] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260809 15:53:07.470381 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:07.411361 14507 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:07.418586 14701 raft_consensus.cc:729] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:07.417178 14638 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:07.472369 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 12 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:07.472955 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:07.425773 14487 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:07.485551 11632 raft_consensus.cc:2454] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:53:07.421262 14652 raft_consensus.cc:2781] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 15:53:07.425559 14642 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:07.413302 14679 consensus_queue.cc:1035] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.059s
I20260809 15:53:07.447983 14655 raft_consensus.cc:482] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:07.473201 11010 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2328) took 1198 ms. Trace:
I20260809 15:53:07.479887 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:07.480559 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:07.529986 11606 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:53:07.481820 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:07.532327 10843 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 8.
I20260809 15:53:07.483353 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:53:07.532755 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 47682.5us
I20260809 15:53:07.487102 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:07.533793 10929 raft_consensus.cc:2379] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 8: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:07.494403 11015 raft_consensus.cc:1226] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 5. Current term is 6. Ops: []
I20260809 15:53:07.444340 11637 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 12 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 12.
I20260809 15:53:07.531448 11010 rpcz_store.cc:276] 0809 15:53:06.275076 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:06.275228 (+ 152us) service_pool.cc:227] Handling call
0809 15:53:06.332204 (+ 56976us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:06.344217 (+ 12013us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:06.344251 (+ 34us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:06.344659 (+ 408us) log.cc:842] Serialized 26 byte log entry
0809 15:53:06.731788 (+387129us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:53:06.731835 (+ 47us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:06.731914 (+ 79us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:07.233799 (+501885us) spinlock_profiling.cc:243] Waited 11.3 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:53:07.447397 (+213598us) raft_consensus.cc:1671] finished
0809 15:53:07.447417 (+ 20us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:07.473118 (+ 25701us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":14136,"raft.run_cpu_time_us":127,"raft.run_wall_time_us":42678,"spinlock_wait_cycles":25924992,"thread_start_us":277173,"threads_started":1,"wal-append.queue_time_us":523976}
I20260809 15:53:07.639261 10978 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:07.640727 10844 raft_consensus.cc:1261] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 7 index: 7. (index mismatch)
I20260809 15:53:07.474220 14658 raft_consensus.cc:2726] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:07.641760 10841 raft_consensus.cc:1226] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 8. Current term is 9. Ops: []
I20260809 15:53:07.418891 14666 consensus_queue.cc:1046] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.7, Next index: 8, Last known committed idx: 6, Time since last communication: 0.059s
I20260809 15:53:07.657420 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:07.648991 10924 raft_consensus.cc:1261] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260809 15:53:07.650192 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 152538us
I20260809 15:53:07.660976 11606 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:53:07.484544 14641 raft_consensus.cc:2741] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
W20260809 15:53:07.651052 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 115117us
I20260809 15:53:07.515121 14706 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:07.492425 14771 raft_consensus.cc:2726] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:53:07.658191 10931 raft_consensus.cc:2454] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:53:07.678959 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:07.680861 10978 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:07.689352 11014 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
W20260809 15:53:07.691596 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 239704us
I20260809 15:53:07.498464 14407 raft_consensus.cc:2726] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:53:07.695576 10978 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:07.664125 14164 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:07.773850 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:07.492740 14507 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:07.498209 14638 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:07.531863 14642 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:07.781325 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:07.530659 14487 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:07.781813 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 85302.1us
I20260809 15:53:07.798831 10978 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:53:07.696466 10811 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.791s (SENT) [suppressed 16 similar messages]
I20260809 15:53:07.777813 11014 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 8.
I20260809 15:53:07.789458 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:07.801244 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:07.801723 10809 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:07.718631 14684 raft_consensus.cc:3032] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 15:53:07.719983 14658 raft_consensus.cc:2781] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:53:07.793954 11637 raft_consensus.cc:2454] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:53:07.697678 14641 raft_consensus.cc:2781] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:07.693267 14473 raft_consensus.cc:482] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:07.820921 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:07.796088 14753 raft_consensus.cc:2726] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:07.804932 10838 raft_consensus.cc:3037] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:07.686939 14733 consensus_queue.cc:1046] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.109s
I20260809 15:53:07.779757 14503 leader_election.cc:288] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:07.835886 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:07.806182 14684 raft_consensus.cc:729] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:07.814039 14655 raft_consensus.cc:3037] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:53:07.849582 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:07.850672 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:07.683156 14769 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:53:07.807746 14754 log.cc:924] Time spent T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.441s user 0.004s sys 0.000s
I20260809 15:53:07.807766 14650 raft_consensus.cc:2781] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election won for term 6
W20260809 15:53:07.807576 14788 log.cc:924] Time spent T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.071s user 0.001s sys 0.000s
I20260809 15:53:07.681982 14701 consensus_queue.cc:260] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 7.5, Last appended by leader: 5, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:07.811647 14652 raft_consensus.cc:686] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 8 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:07.810400 14658 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:07.808041 14779 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:07.795982 14648 consensus_queue.cc:1035] T 34001ced69f34c6f930f5218eef46524 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260809 15:53:07.812615 14641 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:07.807720 14784 raft_consensus.cc:2781] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Leader election won for term 10
I20260809 15:53:07.929437 14641 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:07.839063 14164 raft_consensus.cc:2781] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 15:53:07.934473 14164 raft_consensus.cc:686] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [term 8 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:07.935844 14507 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:07.935741 14164 consensus_queue.cc:237] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 6.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:07.913458 11606 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:07.916949 14407 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:07.921195 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:07.895574 14650 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:07.929623 14784 raft_consensus.cc:686] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [term 10 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:07.883229 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:07.874722 14728 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:07.807734 14764 consensus_queue.cc:1035] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.441s
I20260809 15:53:07.807777 14760 consensus_queue.cc:1035] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.436s
I20260809 15:53:07.940280 14638 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:07.903301 14658 raft_consensus.cc:3037] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:07.944048 13973 raft_consensus.cc:3032] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:53:07.946218 14642 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:07.997568 14005 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:08.005117 14650 raft_consensus.cc:3037] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:07.950825 14769 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:07.893362 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:07.917670 14787 raft_consensus.cc:2781] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election won for term 8
I20260809 15:53:07.824451 14473 raft_consensus.cc:504] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:07.927398 14487 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:53:07.984115 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 70766.9us
W20260809 15:53:07.998922 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 200180us
I20260809 15:53:08.000514 11636 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 10.
W20260809 15:53:08.004819 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 308356us
I20260809 15:53:08.009773 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
W20260809 15:53:07.998574 14048 consensus_meta.cc:287] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.711s user 0.002s sys 0.011s
W20260809 15:53:08.031049 10893 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.557s user 0.000s sys 0.017s
W20260809 15:53:08.109994 10893 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.636s user 0.000s sys 0.017s
W20260809 15:53:08.110391 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 645138us
I20260809 15:53:08.111153 10893 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:53:07.948139 14684 consensus_queue.cc:260] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.098245 14612 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:08.113543 11606 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
W20260809 15:53:08.107488 10811 leader_election.cc:334] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 2.188s (SENT)
I20260809 15:53:08.004000 14407 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.031378 13973 raft_consensus.cc:729] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:08.122978 10811 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:08.134840 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:08.135293 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:08.135893 10841 raft_consensus.cc:2454] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:53:08.124764 11641 raft_consensus.cc:2454] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
W20260809 15:53:08.110150 14527 consensus_meta.cc:287] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.859s user 0.010s sys 0.000s
I20260809 15:53:08.134738 10848 raft_consensus.cc:1226] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: []
W20260809 15:53:08.132043 10840 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.639s user 0.002s sys 0.009s
W20260809 15:53:08.144618 10840 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.651s user 0.002s sys 0.009s
I20260809 15:53:07.964371 14779 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.152437 10841 raft_consensus.cc:1261] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 15:53:08.154925 11018 raft_consensus.cc:1261] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
W20260809 15:53:08.157760 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 50298.4us
I20260809 15:53:08.111725 14787 raft_consensus.cc:686] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:08.161535 10893 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:53:08.138429 14638 raft_consensus.cc:2781] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:53:08.031001 14728 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.094913 14005 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.172724 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:08.175468 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:08.177150 11641 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 12 FOLLOWER]: Advancing to term 13
W20260809 15:53:08.179195 11604 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.648s user 0.005s sys 0.009s
W20260809 15:53:08.180401 11604 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.682s user 0.005s sys 0.009s
W20260809 15:53:08.173734 10847 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.696s user 0.012s sys 0.002s
I20260809 15:53:08.156267 14455 raft_consensus.cc:2781] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 15:53:08.181988 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:53:08.174052 10922 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.677s user 0.000s sys 0.013s
W20260809 15:53:08.186141 10922 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.691s user 0.001s sys 0.013s
W20260809 15:53:08.183621 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 685357us
W20260809 15:53:08.183928 10847 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.706s user 0.012s sys 0.002s
I20260809 15:53:08.190452 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9c38f65ac0de428ba1d25c6f0bb6dc91" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:08.196409 11639 raft_consensus.cc:3037] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:08.183290 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:08.182729 14625 raft_consensus.cc:2781] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:53:08.213807 11641 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 13.
I20260809 15:53:08.110980 14652 consensus_queue.cc:237] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 6.5, Last appended by leader: 5, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:08.213967 11632 raft_consensus.cc:2454] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:53:08.185464 14455 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:08.167335 14648 consensus_queue.cc:1046] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 5.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.031s
W20260809 15:53:08.185583 14674 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.524s user 0.007s sys 0.007s
I20260809 15:53:08.178648 14679 consensus_queue.cc:1035] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.042s
I20260809 15:53:08.209115 14769 raft_consensus.cc:2726] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 15:53:08.154140 14784 consensus_queue.cc:237] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 7.7, Last appended by leader: 7, Current term: 10, Majority size: 3, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.248183 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:53:08.231208 14674 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.580s user 0.008s sys 0.007s
I20260809 15:53:08.247037 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:08.249611 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:53:08.266337 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 127095us
I20260809 15:53:08.227432 14455 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Advancing to term 10
I20260809 15:53:08.220144 14796 raft_consensus.cc:2726] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
W20260809 15:53:08.253252 14802 log.cc:924] Time spent T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.108s user 0.001s sys 0.005s
I20260809 15:53:08.183060 14805 consensus_queue.cc:1035] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.090s
W20260809 15:53:08.324038 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 197602us
I20260809 15:53:08.320163 11015 raft_consensus.cc:2379] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 6: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:08.185145 13973 consensus_queue.cc:260] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.257053 14771 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:08.362200 14407 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:08.362082 14561 consensus_queue.cc:1035] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 15:53:08.366868 10978 leader_election.cc:302] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:08.367609 14407 raft_consensus.cc:2726] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:53:08.369382 10978 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:08.289916 14473 leader_election.cc:288] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:08.379334 14779 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:08.379918 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:08.381088 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:08.381022 14641 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.381911 11640 raft_consensus.cc:2379] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 8: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
W20260809 15:53:08.322889 14799 log.cc:924] Time spent T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.314s user 0.001s sys 0.000s
W20260809 15:53:08.386121 10838 consensus_meta.cc:287] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.549s user 0.011s sys 0.000s
W20260809 15:53:08.387974 14814 log.cc:924] Time spent T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.060s user 0.000s sys 0.000s
I20260809 15:53:08.315824 14674 raft_consensus.cc:482] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:08.390600 10838 raft_consensus.cc:2454] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
W20260809 15:53:08.399456 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 204926us
I20260809 15:53:08.455443 10893 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:08.238310 14798 consensus_queue.cc:1035] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.106s
I20260809 15:53:08.460708 10893 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:08.384837 14612 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:08.465694 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
W20260809 15:53:08.466866 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 97549.2us
I20260809 15:53:08.478979 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 4 index: 2 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:53:08.456055 14794 log.cc:924] Time spent T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.318s user 0.001s sys 0.000s
I20260809 15:53:08.502429 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:08.509394 11640 raft_consensus.cc:2396] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 8 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260809 15:53:08.461989 14706 consensus_meta.cc:287] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.785s user 0.012s sys 0.000s
I20260809 15:53:08.516598 10924 raft_consensus.cc:1226] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 13 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 12. Current term is 13. Ops: []
I20260809 15:53:08.518890 10978 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:08.520200 10978 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:08.504420 10893 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:08.521919 10893 leader_election.cc:302] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:08.497052 14638 raft_consensus.cc:686] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:53:08.464313 14679 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:53:08.536243 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:08.536980 10927 raft_consensus.cc:3037] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:08.534811 11640 raft_consensus.cc:1226] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:08.541141 11640 raft_consensus.cc:1226] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 5. Current term is 6. Ops: []
I20260809 15:53:08.499976 14407 raft_consensus.cc:2726] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:53:08.498919 14625 raft_consensus.cc:686] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 6 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:08.548084 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:08.510807 14779 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:08.454651 14771 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.265079 14787 consensus_queue.cc:237] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.553815 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:08.560422 10924 raft_consensus.cc:2379] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 8: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:08.533806 14521 raft_consensus.cc:2726] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
I20260809 15:53:08.461895 14674 raft_consensus.cc:504] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.566627 10923 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 901) took 1147 ms. Trace:
I20260809 15:53:08.594220 14728 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:53:08.605235 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 83419.9us
I20260809 15:53:08.506806 14612 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.598611 10923 rpcz_store.cc:276] 0809 15:53:07.419084 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:07.419245 (+ 161us) service_pool.cc:227] Handling call
0809 15:53:07.419536 (+ 291us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:07.438972 (+ 19436us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:07.439016 (+ 44us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:07.439507 (+ 491us) log.cc:842] Serialized 26 byte log entry
0809 15:53:07.888180 (+448673us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:53:08.235383 (+347203us) log.cc:842] Serialized 14 byte log entry
0809 15:53:08.235660 (+ 277us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:08.235748 (+ 88us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:08.566350 (+330602us) raft_consensus.cc:1671] finished
0809 15:53:08.566377 (+ 27us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:08.566537 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":8299,"prepare.queue_time_us":401295,"prepare.run_cpu_time_us":36,"prepare.run_wall_time_us":37,"raft.queue_time_us":223393,"raft.run_cpu_time_us":130,"raft.run_wall_time_us":27331,"thread_start_us":536419,"threads_started":2,"wal-append.queue_time_us":662885}
I20260809 15:53:08.606022 10893 leader_election.cc:302] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:08.554288 14005 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:53:08.625124 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 307747us
I20260809 15:53:08.560946 14407 raft_consensus.cc:2781] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:08.636857 10811 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:08.649223 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:08.547025 14679 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:53:08.650900 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 44964.3us
I20260809 15:53:08.671418 11640 raft_consensus.cc:1261] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 4 index: 6. (index mismatch)
W20260809 15:53:08.635172 14675 consensus_meta.cc:287] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 1.167s user 0.010s sys 0.002s
I20260809 15:53:08.672531 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:08.678355 11639 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 8.
I20260809 15:53:08.680563 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:53:08.686625 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 49880.6us
I20260809 15:53:08.687634 10811 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:08.559863 14779 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:08.701333 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:08.684337 11640 raft_consensus.cc:2454] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 8.
W20260809 15:53:08.701808 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 376393us
I20260809 15:53:08.702775 11640 raft_consensus.cc:1226] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 13 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 12. Current term is 13. Ops: []
I20260809 15:53:08.700994 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:08.641587 14407 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:08.652380 14827 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:08.707062 11011 raft_consensus.cc:1226] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 5. Current term is 6. Ops: []
I20260809 15:53:08.713202 11636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:08.718456 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:08.723484 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:08.724123 10840 raft_consensus.cc:2454] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 7.
I20260809 15:53:08.725157 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:08.734161 14789 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:08.725878 10840 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
W20260809 15:53:08.734161 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 46615.3us
I20260809 15:53:08.735448 10811 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:08.737203 14789 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:08.739490 14789 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:08.711198 14641 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:08.715466 14407 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
W20260809 15:53:08.775122 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 106185us
I20260809 15:53:08.725766 14800 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:53:08.726752 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 206631us
I20260809 15:53:08.733197 14473 raft_consensus.cc:2726] T 9c38f65ac0de428ba1d25c6f0bb6dc91 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:53:08.786761 14854 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:08.787732 14674 leader_election.cc:288] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:08.690883 14675 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:08.789274 14819 log.cc:924] Time spent T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.198s user 0.000s sys 0.000s
W20260809 15:53:08.790017 14816 log.cc:924] Time spent T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.239s user 0.000s sys 0.000s
I20260809 15:53:08.726748 14827 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:08.826275 14847 consensus_queue.cc:1035] T 1faddf7ecc6e48ba9306d7ece1f936b1 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.041s
I20260809 15:53:08.784686 14706 raft_consensus.cc:2726] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
W20260809 15:53:08.790946 14818 log.cc:924] Time spent T e6e09d263f96440a99c99d5a5c6ea477 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.158s user 0.000s sys 0.000s
I20260809 15:53:08.828171 14706 raft_consensus.cc:2781] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:08.828269 14827 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:08.828577 14706 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:53:08.829054 14706 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:08.823377 14675 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:08.790794 14800 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:08.830125 10838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 877) took 1366 ms. Trace:
I20260809 15:53:08.830499 10838 rpcz_store.cc:276] 0809 15:53:07.463515 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:07.470029 (+ 6514us) service_pool.cc:227] Handling call
0809 15:53:08.829927 (+1359898us) spinlock_profiling.cc:243] Waited 334 ms on lock 0x7b54000bf4d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:08.830029 (+ 102us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":767669120}
I20260809 15:53:08.791498 14779 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:08.791908 14831 raft_consensus.cc:2726] T 974451500a6e434f8d407d9f178ab710 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:53:08.792253 14771 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:08.796262 10927 raft_consensus.cc:2454] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
W20260809 15:53:08.778334 14640 consensus_meta.cc:287] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 1.436s user 0.008s sys 0.005s
I20260809 15:53:08.792265 14679 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:08.790293 14803 raft_consensus.cc:2726] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260809 15:53:08.804572 14655 consensus_meta.cc:287] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.915s user 0.008s sys 0.004s
I20260809 15:53:08.849378 11634 raft_consensus.cc:2454] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:53:08.793072 14812 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:08.816322 14854 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.823344 11636 raft_consensus.cc:1261] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260809 15:53:08.884217 14787 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:08.884737 14787 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.885176 10840 raft_consensus.cc:2454] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 8.
I20260809 15:53:08.705834 14638 consensus_queue.cc:237] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.710561 14625 consensus_queue.cc:237] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.860236 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:53:08.882396 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 84385us
I20260809 15:53:08.814893 14674 consensus_queue.cc:1046] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.007s
I20260809 15:53:08.887846 10838 raft_consensus.cc:2454] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 8.
I20260809 15:53:08.888247 14674 raft_consensus.cc:3032] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 12 LEADER]: Stepping down as leader of term 12
I20260809 15:53:08.888547 10978 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:53:08.888622 14674 raft_consensus.cc:729] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 12 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:08.889045 10838 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb"
dest_uuid: "a53441872c0c416b8e2f26f52399f536"
mode: GRACEFUL
new_leader_uuid: "20730d75ad954a82b126148f8152a9f3"
from {username='slave'} at 127.0.0.1:49776
I20260809 15:53:08.889670 10838 raft_consensus.cc:595] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Received request to transfer leadership to TS 20730d75ad954a82b126148f8152a9f3
I20260809 15:53:08.890072 10838 raft_consensus.cc:603] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 15:53:08.889796 14674 consensus_queue.cc:260] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 12.11, Last appended by leader: 11, Current term: 12, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.891496 14674 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 12 FOLLOWER]: Advancing to term 13
W20260809 15:53:08.891218 11604 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.621s user 0.000s sys 0.014s
W20260809 15:53:08.896119 11604 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.636s user 0.000s sys 0.015s
I20260809 15:53:08.897567 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:08.901014 14771 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:08.901958 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:08.901660 14771 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.902880 11009 raft_consensus.cc:2379] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 8: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
W20260809 15:53:08.907045 14821 log.cc:924] Time spent T 00000000000000000000000000000000 P 9adabcb74b524f959359b60ec0f8486f: Append to log took a long time: real 0.257s user 0.001s sys 0.000s
I20260809 15:53:08.916704 10840 raft_consensus.cc:1226] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:08.912696 11606 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:53:08.836017 14521 consensus_queue.cc:1046] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.058s
I20260809 15:53:08.880128 14679 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:08.924985 11636 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f"
dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34950
I20260809 15:53:08.925410 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ff6a4ed964b47dba2d0ea0a114e97f4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:08.892928 14787 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:08.933192 14864 raft_consensus.cc:2781] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:08.933316 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:08.893579 14779 raft_consensus.cc:482] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:53:08.933681 14455 consensus_meta.cc:287] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.610s user 0.012s sys 0.005s
I20260809 15:53:08.935516 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:08.938898 10838 raft_consensus.cc:2379] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 7: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
W20260809 15:53:08.923869 14658 consensus_meta.cc:287] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.831s user 0.012s sys 0.000s
W20260809 15:53:08.953289 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 203781us
I20260809 15:53:08.955155 14771 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:08.955472 14857 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:53:08.936870 14745 log.cc:924] Time spent T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 1.863s user 0.006s sys 0.003s
I20260809 15:53:08.973218 11637 raft_consensus.cc:2379] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 10: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:53:08.947328 14787 consensus_queue.cc:1046] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 6.4, Next index: 5, Last known committed idx: 4, Time since last communication: 0.008s
W20260809 15:53:08.956300 14808 log.cc:924] Time spent T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.316s user 0.000s sys 0.001s
I20260809 15:53:08.956228 14841 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:08.878947 14655 raft_consensus.cc:504] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.920007 14612 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:08.883404 14812 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:08.975873 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 85564.5us
I20260809 15:53:08.907691 14783 consensus_queue.cc:1046] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.099s
I20260809 15:53:08.976343 10811 leader_election.cc:302] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:08.982818 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:08.984421 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:08.938421 14864 raft_consensus.cc:482] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:09.000509 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.016678 11009 raft_consensus.cc:2396] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 9 because replica has last-logged OpId of term: 8 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 5.
W20260809 15:53:09.016727 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 44361us
W20260809 15:53:08.982020 14810 log.cc:924] Time spent T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.378s user 0.000s sys 0.001s
I20260809 15:53:08.977967 14394 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:08.977159 14800 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:09.024662 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 3 index: 2 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:09.024848 10811 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:09.026490 10926 raft_consensus.cc:1226] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5. Ops: []
I20260809 15:53:09.034986 10811 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:53:08.939632 14679 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.970489 14779 raft_consensus.cc:504] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:08.968063 14455 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Starting leader election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.016324 14864 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:09.047243 11009 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.059255 14861 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:09.059851 11009 raft_consensus.cc:2379] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 8: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:08.981154 14857 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.059815 14861 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.061942 14857 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:09.062143 14874 raft_consensus.cc:2781] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:09.049703 14868 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:09.062963 14868 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.027101 14787 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:09.064419 14787 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.065346 14868 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:09.057296 14825 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:09.062646 14874 raft_consensus.cc:482] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:09.066552 14787 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:09.066372 14825 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:09.066970 14871 raft_consensus.cc:3032] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:53:09.067396 14871 raft_consensus.cc:729] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
W20260809 15:53:09.068141 14829 log.cc:924] Time spent T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.093s user 0.000s sys 0.000s
I20260809 15:53:09.068699 14825 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:09.069312 14825 raft_consensus.cc:2726] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:53:09.068363 14871 consensus_queue.cc:260] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 6.4, Last appended by leader: 4, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:09.069540 14851 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:09.066581 14874 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:09.057420 14859 consensus_queue.cc:1046] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 15:53:09.058988 14783 raft_consensus.cc:2726] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:53:09.017297 14841 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.058835 14812 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:08.980389 14658 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.074412 11636 raft_consensus.cc:595] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:09.101272 11636 raft_consensus.cc:603] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 15:53:09.055845 10841 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 10.
I20260809 15:53:09.082777 10811 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:09.103547 14783 raft_consensus.cc:2726] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:09.104527 14783 consensus_queue.cc:1046] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260809 15:53:09.105579 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:09.106359 10927 raft_consensus.cc:3037] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:09.111966 10923 raft_consensus.cc:1226] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [term 6 LEADER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 5. Current term is 6. Ops: []
I20260809 15:53:09.116199 10923 raft_consensus.cc:1226] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [term 6 LEADER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 5. Current term is 6. Ops: []
I20260809 15:53:09.117609 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:09.118542 10923 raft_consensus.cc:2379] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 6: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
W20260809 15:53:09.129140 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 217330us
I20260809 15:53:09.020051 14612 consensus_queue.cc:1035] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260809 15:53:09.137897 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:53:09.139164 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 120617us
I20260809 15:53:09.139796 11014 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 8.
I20260809 15:53:09.141089 10809 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:09.144081 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:09.144742 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:09.031963 14394 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.147311 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:09.150535 10923 raft_consensus.cc:2454] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:53:09.150707 10927 raft_consensus.cc:2454] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 8.
W20260809 15:53:09.130061 14766 consensus_meta.cc:287] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 1.686s user 0.017s sys 0.000s
I20260809 15:53:09.148923 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:09.151930 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.159789 11017 raft_consensus.cc:2379] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 6: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
W20260809 15:53:09.149222 14810 log.cc:924] Time spent T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.050s user 0.000s sys 0.000s
I20260809 15:53:09.170624 11635 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41478 (request call id 1082) took 1450 ms. Trace:
I20260809 15:53:09.132928 14871 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:09.160059 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:09.155839 11606 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters:
W20260809 15:53:09.163494 14838 log.cc:924] Time spent T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.068s user 0.001s sys 0.000s
I20260809 15:53:09.164945 14841 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:09.171001 11635 rpcz_store.cc:276] 0809 15:53:07.719722 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:07.720542 (+ 820us) service_pool.cc:227] Handling call
0809 15:53:07.720984 (+ 442us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:07.818971 (+ 97987us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:07.819015 (+ 44us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:07.834929 (+ 15914us) log.cc:842] Serialized 26 byte log entry
0809 15:53:08.139038 (+304109us) raft_consensus.cc:1631] Marking committed up to 9
0809 15:53:08.161245 (+ 22207us) log.cc:842] Serialized 14 byte log entry
0809 15:53:08.161414 (+ 169us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:08.161474 (+ 60us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:09.170433 (+1008959us) raft_consensus.cc:1671] finished
0809 15:53:09.170451 (+ 18us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:09.170569 (+ 118us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":22441,"prepare.run_cpu_time_us":33,"prepare.run_wall_time_us":33,"raft.queue_time_us":32873,"raft.run_cpu_time_us":241,"raft.run_wall_time_us":23417,"thread_start_us":242734,"threads_started":1,"wal-append.queue_time_us":441431}
I20260809 15:53:09.165788 14655 leader_election.cc:288] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:09.150959 14455 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:09.177076 10930 raft_consensus.cc:2454] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:53:09.178584 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:09.149034 14859 raft_consensus.cc:2781] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 15:53:09.186983 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6e09d263f96440a99c99d5a5c6ea477" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:09.226325 14859 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:09.167850 14394 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:09.200578 10811 connection.cc:232] RPC call timeout handler was delayed by -1.3674415129935369s: this may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow extra 0.18180000000000002s for a response
I20260809 15:53:09.135409 14854 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:09.208817 14842 raft_consensus.cc:2741] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260809 15:53:09.070103 14851 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:09.211655 14855 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:09.217443 14872 raft_consensus.cc:482] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:09.157470 11604 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:09.218952 14861 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:09.219900 14779 leader_election.cc:288] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:09.220642 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 218540us
W20260809 15:53:09.220588 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 64825.5us
I20260809 15:53:09.173461 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:09.222754 10920 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 898) took 1886 ms. Trace:
I20260809 15:53:09.223304 11642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 829) took 1778 ms. Trace:
I20260809 15:53:09.241900 10920 rpcz_store.cc:276] 0809 15:53:07.336150 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:07.336274 (+ 124us) service_pool.cc:227] Handling call
0809 15:53:07.336531 (+ 257us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:07.492144 (+155613us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:07.492173 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:07.492212 (+ 39us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:53:07.492241 (+ 29us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:09.222514 (+1730273us) spinlock_profiling.cc:243] Waited 155 ms on lock 0x7b5400189f54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:09.222563 (+ 49us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:09.222697 (+ 134us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":357483776}
I20260809 15:53:09.226774 14859 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Advancing to term 10
I20260809 15:53:09.230882 10811 leader_election.cc:302] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:53:09.240840 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 83530.6us
I20260809 15:53:09.241961 11642 rpcz_store.cc:276] 0809 15:53:07.445059 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:07.480436 (+ 35377us) service_pool.cc:227] Handling call
0809 15:53:07.480770 (+ 334us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:07.811592 (+330822us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:07.811632 (+ 40us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:07.811699 (+ 67us) raft_consensus.cc:1631] Marking committed up to 7
0809 15:53:07.811750 (+ 51us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:09.223087 (+1411337us) spinlock_profiling.cc:243] Waited 331 ms on lock 0x7b5400631854. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:09.223135 (+ 48us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:09.223246 (+ 111us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":760288640}
I20260809 15:53:09.225414 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:09.247956 10811 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:09.231174 14847 raft_consensus.cc:2726] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 15:53:09.253157 14877 raft_consensus.cc:2781] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:53:09.253604 14877 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:09.235953 14855 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.253950 14877 raft_consensus.cc:3037] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:53:09.255532 14855 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:09.257104 14247 consensus_queue.cc:260] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.275836 11017 raft_consensus.cc:1261] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 6 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260809 15:53:09.276527 14247 raft_consensus.cc:3037] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:09.258385 14887 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:09.277334 14887 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.278677 11016 raft_consensus.cc:1226] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 5. Current term is 6. Ops: []
I20260809 15:53:09.278980 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.279788 11014 raft_consensus.cc:2379] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 7: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:09.231441 14394 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:09.260617 14864 raft_consensus.cc:504] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:09.285322 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.243985 14872 raft_consensus.cc:504] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.235321 14851 leader_election.cc:288] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:09.250579 14842 raft_consensus.cc:2726] T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:09.288430 11015 raft_consensus.cc:1226] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:09.250389 14261 raft_consensus.cc:2726] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:09.272411 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:09.285950 14864 leader_election.cc:288] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:09.264850 14853 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:09.290880 14874 raft_consensus.cc:504] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.291224 14853 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:09.293848 14783 consensus_queue.cc:1035] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s
I20260809 15:53:09.294956 14706 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.295229 14783 consensus_queue.cc:1046] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:53:09.274596 14700 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:09.296948 14706 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:09.245045 14640 raft_consensus.cc:2726] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
W20260809 15:53:09.276647 14782 log.cc:924] Time spent T fe21e0688ade405bb4590b9c256bd696 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 1.599s user 0.003s sys 0.000s
I20260809 15:53:09.298662 11633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 825) took 1857 ms. Trace:
I20260809 15:53:09.299044 11633 rpcz_store.cc:276] 0809 15:53:07.441534 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:07.465890 (+ 24356us) service_pool.cc:227] Handling call
0809 15:53:07.504407 (+ 38517us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:07.809706 (+305299us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:07.809754 (+ 48us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:07.809812 (+ 58us) raft_consensus.cc:1631] Marking committed up to 7
0809 15:53:07.809862 (+ 50us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:09.298455 (+1488593us) spinlock_profiling.cc:243] Waited 305 ms on lock 0x7b54004f3dd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:09.298499 (+ 44us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:09.298612 (+ 113us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":701530368}
I20260809 15:53:09.300235 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:53:09.302207 14780 log.cc:924] Time spent T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 1.609s user 0.001s sys 0.001s
I20260809 15:53:09.258505 14866 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:53:09.303232 14650 consensus_meta.cc:287] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 1.199s user 0.022s sys 0.000s
I20260809 15:53:09.302902 14866 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:09.303653 14650 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.305033 11641 raft_consensus.cc:1226] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260809 15:53:09.307003 14877 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.310478 14402 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Advancing to term 10
I20260809 15:53:09.311299 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:09.311407 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:53:09.294056 11639 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41478 (request call id 1085) took 1479 ms (client timeout 1667 ms). Trace:
W20260809 15:53:09.316079 11639 rpcz_store.cc:269] 0809 15:53:07.815127 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:07.815251 (+ 124us) service_pool.cc:227] Handling call
0809 15:53:09.293865 (+1478614us) spinlock_profiling.cc:243] Waited 317 ms on lock 0x7b54004e7fd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:09.293991 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":730131200}
I20260809 15:53:09.296409 10841 raft_consensus.cc:1261] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260809 15:53:09.298292 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:09.317536 10842 raft_consensus.cc:1261] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
W20260809 15:53:09.310601 14407 consensus_meta.cc:287] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.517s user 0.005s sys 0.006s
W20260809 15:53:09.311316 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 59746.2us
I20260809 15:53:09.312373 14261 consensus_queue.cc:1035] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 15:53:09.318688 14407 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.319288 11018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2545) took 1352 ms. Trace:
I20260809 15:53:09.319641 11018 rpcz_store.cc:276] 0809 15:53:07.967206 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:07.997243 (+ 30037us) service_pool.cc:227] Handling call
0809 15:53:07.997602 (+ 359us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:08.155292 (+157690us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:09.319056 (+1163764us) spinlock_profiling.cc:243] Waited 157 ms on lock 0x7b54001ddb54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:09.319232 (+ 176us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":361044224}
I20260809 15:53:09.320904 14254 raft_consensus.cc:3032] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 15:53:09.321878 14254 raft_consensus.cc:729] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:53:09.323226 11606 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:53:09.322858 14254 consensus_queue.cc:260] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.321296 11604 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:09.320636 10811 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:09.324736 11606 leader_election.cc:302] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:53:09.283959 14394 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:09.187266 14766 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 13 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.220412 14658 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:09.328874 11639 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:09.330197 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:09.332641 11635 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 10.
I20260809 15:53:09.334096 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:09.335307 10978 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:09.324761 14254 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:09.337752 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:09.347209 11633 raft_consensus.cc:2379] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 5: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:09.321184 14679 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:09.338960 10978 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:53:09.339282 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:09.339759 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:09.357733 10841 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:53:09.358250 10978 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:53:09.358393 14862 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:09.358661 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:09.359404 10841 raft_consensus.cc:2379] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 8: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:09.359011 14862 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.359675 10978 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:53:09.331255 14640 raft_consensus.cc:2726] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:09.360109 14612 raft_consensus.cc:2781] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:53:09.361114 14862 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:09.361728 14394 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:09.361888 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.305037 14866 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:09.313184 14859 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:09.338603 10843 raft_consensus.cc:1226] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 13 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 12. Current term is 13. Ops: []
I20260809 15:53:09.343470 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.326915 14697 raft_consensus.cc:2741] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election decision vote started in defunct term 7: won
I20260809 15:53:09.355912 10811 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:09.288038 14872 leader_election.cc:288] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:09.356132 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:09.361202 14846 raft_consensus.cc:2781] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election won for term 5
W20260809 15:53:09.450130 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 94315.4us
I20260809 15:53:09.362553 11016 raft_consensus.cc:2454] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 4.
I20260809 15:53:09.362819 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:09.361145 10978 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:09.360118 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:09.331401 14650 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:09.363823 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "049fe6a2ea4f4a15a98b9c6c5755173d" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:09.342101 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.454116 10841 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:09.454555 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.454851 10811 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:09.280160 14887 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:09.366591 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "08f368b4f4e748c596049c81eaed3f70" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.361987 14876 raft_consensus.cc:2781] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 15:53:09.369556 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:09.370244 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:09.320917 14407 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:09.293892 14874 leader_election.cc:288] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:09.355751 14503 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:09.374106 14888 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:09.374533 14873 raft_consensus.cc:2781] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 15:53:09.336503 14487 raft_consensus.cc:482] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:09.361323 14784 consensus_queue.cc:1035] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.014s
I20260809 15:53:09.379765 14770 raft_consensus.cc:3032] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:53:09.385468 10929 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 2164) took 1504 ms. Trace:
I20260809 15:53:09.500608 10929 rpcz_store.cc:276] 0809 15:53:07.880648 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:07.881280 (+ 632us) service_pool.cc:227] Handling call
0809 15:53:07.881585 (+ 305us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:08.109623 (+228038us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:08.109656 (+ 33us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:08.109691 (+ 35us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:53:08.109732 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:09.385237 (+1275505us) spinlock_profiling.cc:243] Waited 228 ms on lock 0x7b5400372154. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:09.385281 (+ 44us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:09.385409 (+ 128us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":523903872}
I20260809 15:53:09.337240 14247 raft_consensus.cc:2781] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:53:09.468459 14876 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:09.501789 14247 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:09.502146 14876 raft_consensus.cc:3037] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Advancing to term 9
W20260809 15:53:09.389052 14882 log.cc:924] Time spent T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.051s user 0.000s sys 0.001s
I20260809 15:53:09.395180 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:09.507297 14848 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:09.507828 14848 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.508183 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:09.508832 10847 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 10.
I20260809 15:53:09.336688 14864 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:09.509650 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:09.398195 10837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.GetConsensusState from 127.0.0.1:45572 (request call id 1017) took 1387 ms. Trace:
W20260809 15:53:09.398415 14884 log.cc:924] Time spent T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.060s user 0.000s sys 0.001s
I20260809 15:53:09.399266 10842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2397) took 1972 ms. Trace:
I20260809 15:53:09.402556 14766 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 13 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:09.403777 14853 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:09.404173 11012 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2341) took 1927 ms. Trace:
I20260809 15:53:09.410210 13973 raft_consensus.cc:3037] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:09.414634 11642 raft_consensus.cc:1261] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
W20260809 15:53:09.405818 11634 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41482 (request call id 819) took 1982 ms (client timeout 1818 ms). Trace:
W20260809 15:53:09.424319 14856 log.cc:924] Time spent T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.447s user 0.002s sys 0.000s
W20260809 15:53:09.445261 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 71557.7us
W20260809 15:53:09.446117 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 79560.6us
I20260809 15:53:09.452077 10840 raft_consensus.cc:2396] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 7 because replica has last-logged OpId of term: 5 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 2 index: 5.
W20260809 15:53:09.452179 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 91099.5us
I20260809 15:53:09.452212 10844 raft_consensus.cc:3037] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:09.363529 11635 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:09.343851 14871 raft_consensus.cc:2726] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 15:53:09.460562 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.461206 14866 consensus_queue.cc:1046] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 5.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 15:53:09.461661 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.447991 14775 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:09.461831 14877 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:09.467108 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:09.574501 11013 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Advancing to term 10
I20260809 15:53:09.574683 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:09.575410 10847 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 4.
I20260809 15:53:09.372071 14521 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:09.478170 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:09.579118 11017 raft_consensus.cc:2396] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 13 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260809 15:53:09.452733 14048 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:09.449853 14612 raft_consensus.cc:686] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:09.492094 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:09.463203 14859 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 10 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:09.498549 14487 raft_consensus.cc:504] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.500120 14770 raft_consensus.cc:729] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:09.502312 14247 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:09.387794 14492 raft_consensus.cc:3032] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:53:09.397423 14674 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:09.510392 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:09.510550 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:09.510980 14730 consensus_queue.cc:1035] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:53:09.511273 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:09.491937 14503 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:09.509874 14864 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:09.512332 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:09.513425 10893 leader_election.cc:302] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:09.514246 14827 consensus_queue.cc:1035] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 15:53:09.475159 14658 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:09.516465 14805 consensus_queue.cc:1046] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:53:09.517416 11641 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 1056) took 1328 ms. Trace:
I20260809 15:53:09.474846 14675 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:09.509835 14848 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:09.478066 14836 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:09.473619 14666 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:09.520001 10837 rpcz_store.cc:276] 0809 15:53:08.010595 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:08.010726 (+ 131us) service_pool.cc:227] Handling call
0809 15:53:09.397560 (+1386834us) spinlock_profiling.cc:243] Waited 144 ms on lock 0x7b540037e1d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478af0 00007f79694fbf2a 00007f796e438c6e 00007f7967b2f807 00007f7967b2f792 00007f7967b2f6c1 00007f7967b2f63c 00007f7967b2e8f2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:09.398093 (+ 533us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":331735808}
I20260809 15:53:09.520709 10842 rpcz_store.cc:276] 0809 15:53:07.427158 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:07.443448 (+ 16290us) service_pool.cc:227] Handling call
0809 15:53:07.477130 (+ 33682us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:09.317961 (+1840831us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:09.399048 (+ 81087us) spinlock_profiling.cc:243] Waited 1.84 s on lock 0x7b54001f09d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:09.399208 (+ 160us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4231763328}
I20260809 15:53:09.523104 10923 raft_consensus.cc:1261] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 5 index: 4. Preceding OpId from leader: term: 8 index: 5. (index mismatch)
I20260809 15:53:09.478366 14887 raft_consensus.cc:2726] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 15:53:09.523524 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:53:09.524714 11634 rpcz_store.cc:269] 0809 15:53:07.423623 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:07.424165 (+ 542us) service_pool.cc:227] Handling call
0809 15:53:09.405620 (+1981455us) spinlock_profiling.cc:243] Waited 1.4 s on lock 0x7b54005212d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:09.405735 (+ 115us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3224928640}
I20260809 15:53:09.522071 11012 rpcz_store.cc:276] 0809 15:53:07.477156 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:07.477376 (+ 220us) service_pool.cc:227] Handling call
0809 15:53:07.533266 (+ 55890us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:07.655778 (+122512us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:07.655820 (+ 42us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:07.655870 (+ 50us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:53:07.655914 (+ 44us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:09.403965 (+1748051us) spinlock_profiling.cc:243] Waited 122 ms on lock 0x7b54003f8c54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:09.404019 (+ 54us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:09.404122 (+ 103us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":281184000}
I20260809 15:53:09.567762 10978 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:53:09.523481 10841 raft_consensus.cc:2454] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 8.
I20260809 15:53:09.497399 14888 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.571980 11606 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:53:09.468734 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.577322 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:09.523746 14638 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:09.581779 11604 leader_election.cc:302] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:09.669018 10924 raft_consensus.cc:2454] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:53:09.702759 10840 raft_consensus.cc:2454] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
W20260809 15:53:09.703956 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 136256us
W20260809 15:53:09.704687 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 122976us
W20260809 15:53:09.704619 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 132694us
I20260809 15:53:09.643622 14801 raft_consensus.cc:2726] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:09.518761 14873 raft_consensus.cc:686] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 8 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:53:09.585050 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:09.647755 10922 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54236 (request call id 2160) took 2422 ms. Trace:
I20260809 15:53:09.667809 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:09.706017 11604 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:53:09.771267 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 65308.7us
I20260809 15:53:09.706492 10978 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:09.770742 11640 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 10.
I20260809 15:53:09.702576 11641 rpcz_store.cc:276] 0809 15:53:08.189333 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:08.253033 (+ 63700us) service_pool.cc:227] Handling call
0809 15:53:08.316468 (+ 63435us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:09.305681 (+989213us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:09.517171 (+211490us) spinlock_profiling.cc:243] Waited 988 ms on lock 0x7b5400694bd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:09.517358 (+ 187us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2272883072}
I20260809 15:53:09.682246 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:09.582410 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:09.649448 14254 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:09.584564 14866 consensus_queue.cc:1225] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer 20730d75ad954a82b126148f8152a9f3 log is divergent from this leader: its last log entry 6.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260809 15:53:09.737578 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:09.589814 14521 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.578919 14775 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.740423 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:09.741518 14864 leader_election.cc:288] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:09.781517 10924 raft_consensus.cc:3037] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Advancing to term 10
I20260809 15:53:09.761147 14846 raft_consensus.cc:686] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:09.659664 14492 raft_consensus.cc:729] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:09.701054 14770 consensus_queue.cc:260] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:09.787878 14770 raft_consensus.cc:3037] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:09.781925 14254 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:09.699100 14891 log.cc:924] Time spent T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.120s user 0.001s sys 0.000s
I20260809 15:53:09.792523 10843 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45578 (request call id 2137) took 1687 ms. Trace:
I20260809 15:53:09.742832 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:09.794293 10842 raft_consensus.cc:2379] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 10: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:53:09.677819 14487 leader_election.cc:288] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:09.797209 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.782334 14894 raft_consensus.cc:2781] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:09.743464 11635 raft_consensus.cc:2362] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election vote request: Already granted yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5. Re-sending same reply.
I20260809 15:53:09.742022 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:09.744273 14888 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:09.737972 14836 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:09.738340 14638 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.747507 14503 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:09.755798 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.738739 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:09.805745 10841 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Advancing to term 10
I20260809 15:53:09.807117 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:09.807567 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.746044 10837 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:09.525786 14455 consensus_queue.cc:1046] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.123s
I20260809 15:53:09.738165 14407 consensus_queue.cc:1035] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.019s
I20260809 15:53:09.756703 14005 raft_consensus.cc:2781] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:53:09.757261 14784 raft_consensus.cc:2781] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election won for term 8
I20260809 15:53:09.819581 14638 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:09.822794 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:09.823279 14784 raft_consensus.cc:686] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:09.823971 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:09.824553 10926 raft_consensus.cc:2379] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 10: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:09.824625 10920 raft_consensus.cc:2454] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 8.
I20260809 15:53:09.679584 14674 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:53:09.758307 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 244749us
I20260809 15:53:09.816912 14005 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:09.778209 14612 consensus_queue.cc:237] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.749929 10838 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 10.
I20260809 15:53:09.739428 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:09.812978 14521 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:09.703088 10929 raft_consensus.cc:2454] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 8.
W20260809 15:53:09.759277 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 53573.8us
I20260809 15:53:09.777621 11015 raft_consensus.cc:3037] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:53:09.839408 10929 raft_consensus.cc:1226] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 5. Current term is 6. Ops: []
I20260809 15:53:09.752866 14848 raft_consensus.cc:2726] T 9ff6a4ed964b47dba2d0ea0a114e97f4 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
W20260809 15:53:09.779093 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 72660us
I20260809 15:53:09.840701 10929 raft_consensus.cc:1261] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 3 index: 2. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260809 15:53:09.766912 10844 raft_consensus.cc:2454] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:53:09.840890 11013 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 10.
I20260809 15:53:09.760439 14374 raft_consensus.cc:2726] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:09.782333 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:09.782672 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:09.783347 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:09.738514 14675 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:09.764428 14685 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:09.768594 10922 rpcz_store.cc:276] 0809 15:53:07.225457 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:07.226063 (+ 606us) service_pool.cc:227] Handling call
0809 15:53:07.226414 (+ 351us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:07.226749 (+ 335us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:07.226777 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:07.227231 (+ 454us) log.cc:842] Serialized 26 byte log entry
0809 15:53:07.444294 (+217063us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:53:09.395956 (+1951662us) spinlock_profiling.cc:243] Waited 4.38 ms on lock 0x7b1800179e00. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f796918282c 00007f79665ad9dc 00007f796f3e4a39 00007f79665ca8bb 00007f79665c8cbf 00007f79665c8155 00007f796cec8167 00007f79694fd807 00007f796950cf0f
0809 15:53:09.396104 (+ 148us) log.cc:842] Serialized 14 byte log entry
0809 15:53:09.593847 (+197743us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:09.594330 (+ 483us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:09.594376 (+ 46us) raft_consensus.cc:1671] finished
0809 15:53:09.647515 (+ 53139us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:09.647701 (+ 186us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":2673,"prepare.queue_time_us":734295,"prepare.run_cpu_time_us":40,"prepare.run_wall_time_us":40,"raft.queue_time_us":53710,"raft.run_cpu_time_us":248,"raft.run_wall_time_us":4140,"spinlock_wait_cycles":10081024,"thread_start_us":1105636,"threads_started":3,"wal-append.queue_time_us":662221,"wal-append.run_cpu_time_us":2882,"wal-append.run_wall_time_us":1273853}
I20260809 15:53:09.737460 14666 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.790508 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "974451500a6e434f8d407d9f178ab710" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:09.792824 10843 rpcz_store.cc:276] 0809 15:53:08.104985 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:08.107796 (+ 2811us) service_pool.cc:227] Handling call
0809 15:53:08.108164 (+ 368us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:09.443398 (+1335234us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:09.792360 (+348962us) spinlock_profiling.cc:243] Waited 1.23 s on lock 0x7b540005a5d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:09.792474 (+ 114us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2829352704}
I20260809 15:53:09.778723 14763 raft_consensus.cc:2726] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:09.741307 11010 raft_consensus.cc:2454] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:53:09.812323 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:53:09.813926 14893 log.cc:924] Time spent T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.220s user 0.001s sys 0.000s
I20260809 15:53:09.814667 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:09.850775 11017 raft_consensus.cc:2379] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 6: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:09.757376 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:09.800843 14894 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:09.831480 10893 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:09.832192 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.833626 10927 raft_consensus.cc:2454] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 10.
I20260809 15:53:09.853415 10893 leader_election.cc:302] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:09.743755 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.837538 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:09.854671 10920 raft_consensus.cc:2379] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 13: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:09.854522 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:09.854776 10893 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:09.855234 10843 raft_consensus.cc:2379] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 6: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:09.739907 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:09.843214 10809 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:09.844835 10844 raft_consensus.cc:1226] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8. Ops: [5.6-7.7]
I20260809 15:53:09.844422 11640 raft_consensus.cc:2454] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
I20260809 15:53:09.829949 14005 raft_consensus.cc:3037] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:09.851598 11013 raft_consensus.cc:1226] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:09.841455 10978 leader_election.cc:302] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:09.858476 11012 raft_consensus.cc:2379] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 10: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:53:09.859733 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:09.856726 10841 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 10.
I20260809 15:53:09.860306 11640 raft_consensus.cc:3037] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:53:09.838183 14854 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:09.857681 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:09.879562 10844 raft_consensus.cc:1261] T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260809 15:53:09.862978 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:09.879081 10843 raft_consensus.cc:1226] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:09.881743 10841 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:09.867874 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:09.873031 10924 raft_consensus.cc:2454] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 10.
I20260809 15:53:09.867532 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: true dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:09.861902 11015 raft_consensus.cc:2454] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:53:09.884724 11604 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:09.870481 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f891fe62df749108fa553cb0661a874" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:09.888708 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:09.888612 14374 consensus_queue.cc:1046] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260809 15:53:09.889000 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:09.889734 14374 raft_consensus.cc:3032] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 15:53:09.891149 10930 raft_consensus.cc:1261] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260809 15:53:09.891340 14666 leader_election.cc:288] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:09.825719 14492 consensus_queue.cc:260] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.894134 14254 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 10 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:09.894622 14854 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:09.895020 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:09.895298 14775 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:09.895659 11641 raft_consensus.cc:3037] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Advancing to term 10
I20260809 15:53:09.896062 14676 raft_consensus.cc:482] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:09.896324 14638 raft_consensus.cc:482] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:09.896549 14676 raft_consensus.cc:504] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.896837 14638 raft_consensus.cc:504] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.898720 14638 leader_election.cc:288] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:09.899364 14473 raft_consensus.cc:3032] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 15:53:09.899737 14473 raft_consensus.cc:729] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:09.865301 14561 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:09.900552 14473 consensus_queue.cc:260] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.901652 14473 raft_consensus.cc:3037] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:09.903359 14675 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:09.903929 14521 raft_consensus.cc:2781] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260809 15:53:09.898552 10978 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:53:09.903856 14247 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:09.904549 14521 raft_consensus.cc:686] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [term 10 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:09.904740 14871 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:09.905242 14871 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.906915 14674 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:09.907218 10978 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:09.907419 14674 raft_consensus.cc:2781] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:53:09.907882 14674 raft_consensus.cc:686] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:09.908344 10978 leader_election.cc:302] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:09.909039 14864 raft_consensus.cc:2726] T 73ac44823c384ed99817c1ba3bba75ea P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:09.909755 14753 raft_consensus.cc:2726] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:09.911034 10978 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:09.911973 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:09.909750 10843 raft_consensus.cc:1226] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: [6.6-6.6]
I20260809 15:53:09.908694 14674 consensus_queue.cc:237] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:09.914562 11640 raft_consensus.cc:2454] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 5.
I20260809 15:53:09.916795 11606 leader_election.cc:302] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:09.915472 11633 raft_consensus.cc:2379] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 5: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:53:09.911355 10927 raft_consensus.cc:3037] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:53:09.919847 11606 leader_election.cc:302] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:09.916234 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.905848 14247 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:09.824151 14784 consensus_queue.cc:237] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 7.7, Last appended by leader: 7, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.921766 11641 raft_consensus.cc:2454] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 10.
I20260809 15:53:09.935684 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 5 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:09.927425 11606 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:09.931394 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.936406 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:09.924437 10893 leader_election.cc:302] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:09.922989 10978 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:53:09.888769 14897 log.cc:924] Time spent T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.072s user 0.000s sys 0.000s
I20260809 15:53:09.857674 14847 raft_consensus.cc:2781] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader election won for term 10
I20260809 15:53:09.937096 11012 raft_consensus.cc:2379] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 8: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:09.937201 11606 leader_election.cc:302] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:53:09.910929 14374 raft_consensus.cc:729] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:53:09.939239 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.851936 14685 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.866641 14858 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:09.939498 14836 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:09.942911 11017 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 8.
I20260809 15:53:09.948218 10893 leader_election.cc:302] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:09.905478 14521 consensus_queue.cc:237] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 9.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.952963 14685 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:09.940042 14888 raft_consensus.cc:2726] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority
I20260809 15:53:09.940907 10922 raft_consensus.cc:2379] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 5: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:09.947480 14697 raft_consensus.cc:2781] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader election won for term 8
I20260809 15:53:09.954895 14394 raft_consensus.cc:2726] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:53:09.955574 10891 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:09.937074 11641 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 10.
I20260809 15:53:09.857535 14455 raft_consensus.cc:2726] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:09.857265 14894 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:09.864456 14407 raft_consensus.cc:2781] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election won for term 5
I20260809 15:53:09.956640 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:09.957027 14407 raft_consensus.cc:482] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting leader election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:09.957350 11641 raft_consensus.cc:2379] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 7: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:09.957422 14407 raft_consensus.cc:3037] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:53:09.942166 14487 raft_consensus.cc:2726] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:09.958153 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:09.959205 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 13 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:09.959882 11641 raft_consensus.cc:2379] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 13: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:09.959441 14005 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.961041 11634 raft_consensus.cc:2379] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 in current term 7: Already voted for candidate c7aa762318cb4e97875333f10679a9f9 in this term.
I20260809 15:53:09.961284 14676 leader_election.cc:288] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:09.944247 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:09.962006 14871 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:09.940320 14675 consensus_queue.cc:1035] T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.030s
I20260809 15:53:09.943367 14864 raft_consensus.cc:2781] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:53:09.926401 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:09.916951 14753 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:09.917714 14638 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:09.950248 14858 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.951193 14658 raft_consensus.cc:2726] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:09.950773 14374 consensus_queue.cc:260] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.843959 14846 consensus_queue.cc:237] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.956308 14394 raft_consensus.cc:2781] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 15:53:09.939939 14254 raft_consensus.cc:2781] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:53:09.844161 14876 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.955984 10893 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:09.922308 14775 raft_consensus.cc:2726] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:53:09.926000 14641 raft_consensus.cc:3037] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:09.930521 10841 raft_consensus.cc:2454] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 8.
I20260809 15:53:09.944314 11606 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:09.949359 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:09.986773 11641 raft_consensus.cc:1226] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:09.985291 10927 raft_consensus.cc:2454] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:53:09.829254 14873 consensus_queue.cc:237] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:09.995020 14048 raft_consensus.cc:2726] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:09.951822 14827 raft_consensus.cc:2726] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 15:53:10.015632 11633 raft_consensus.cc:2454] T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
W20260809 15:53:10.011600 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 47604.2us
I20260809 15:53:09.978066 14847 raft_consensus.cc:686] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [term 10 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
W20260809 15:53:10.032820 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 88551.6us
I20260809 15:53:09.997776 14697 raft_consensus.cc:686] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 8 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
W20260809 15:53:10.036629 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 80738.3us
I20260809 15:53:10.037310 10893 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters:
I20260809 15:53:10.038190 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:10.039148 10844 raft_consensus.cc:2379] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 10: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:09.999491 14864 raft_consensus.cc:686] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:10.041244 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ac44823c384ed99817c1ba3bba75ea" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:10.042488 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6504ce1c0d4241bfb97eb553c8df35" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 10 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:10.010901 14561 raft_consensus.cc:2726] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:10.044829 10926 raft_consensus.cc:2379] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 10: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:09.948529 14900 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.084s
I20260809 15:53:10.005201 14394 raft_consensus.cc:482] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:10.077618 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:10.078470 11606 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:53:10.095274 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 157105us
I20260809 15:53:10.107537 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:10.081090 10847 raft_consensus.cc:1261] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 5 index: 6. Preceding OpId from leader: term: 6 index: 7. (index mismatch)
I20260809 15:53:10.083928 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:09.994369 14753 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:10.046440 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:09.997427 14638 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:10.106524 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:10.121654 11606 leader_election.cc:302] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:10.046408 11016 raft_consensus.cc:1226] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 8. Current term is 9. Ops: [8.8-8.8]
I20260809 15:53:10.115554 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:10.115919 10978 leader_election.cc:302] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:10.125371 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:10.044858 14676 raft_consensus.cc:3037] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:10.128203 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:10.077965 14812 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:10.150434 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "220f58b662044f089f2be802ed24284a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:10.130762 10927 raft_consensus.cc:1261] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 15:53:10.131673 14826 raft_consensus.cc:2726] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:53:10.044732 14801 raft_consensus.cc:2726] T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:53:10.130015 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:10.040872 14254 raft_consensus.cc:686] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [term 5 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:53:10.121127 14561 raft_consensus.cc:482] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:10.137702 14876 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:10.138443 14612 raft_consensus.cc:2781] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election won for term 8
I20260809 15:53:10.139070 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:10.138950 14407 raft_consensus.cc:504] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:10.168694 14407 leader_election.cc:288] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:10.140316 14787 consensus_queue.cc:1046] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:53:10.120508 14775 consensus_queue.cc:1046] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 7.9, Next index: 10, Last known committed idx: 9, Time since last communication: 0.042s
I20260809 15:53:10.089147 14654 raft_consensus.cc:2781] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Leader election won for term 10
I20260809 15:53:10.169939 14787 raft_consensus.cc:3032] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:53:10.170447 14787 raft_consensus.cc:729] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:10.151405 11641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:10.152661 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:10.115511 14394 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Advancing to term 9
W20260809 15:53:10.127314 13973 consensus_meta.cc:287] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.601s user 0.007s sys 0.003s
I20260809 15:53:10.120620 14822 raft_consensus.cc:2726] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:10.130504 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:10.159170 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:10.129611 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a5ecb5bb6e347a6921a7c894853e92e" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 3 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:10.161311 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:10.163571 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:10.135118 14763 consensus_queue.cc:1035] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.037s
I20260809 15:53:10.120756 14730 consensus_queue.cc:1046] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 2.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.042s
I20260809 15:53:10.166055 10924 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:10.226318 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:10.228936 11636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:34950 (request call id 2) took 1317 ms. Trace:
I20260809 15:53:10.233937 11633 raft_consensus.cc:2396] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 9 because replica has last-logged OpId of term: 7 index: 7, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 6.
W20260809 15:53:10.223348 10921 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54224 (request call id 2156) took 2756 ms (client timeout 1677 ms). Trace:
W20260809 15:53:10.234683 10921 rpcz_store.cc:269] 0809 15:53:07.463738 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:07.463862 (+ 124us) service_pool.cc:227] Handling call
0809 15:53:10.219651 (+2755789us) spinlock_profiling.cc:243] Waited 2.12 s on lock 0x7b540021a7d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:10.219820 (+ 169us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4877318016}
I20260809 15:53:10.217175 11013 raft_consensus.cc:1261] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 15:53:10.234059 11636 rpcz_store.cc:276] 0809 15:53:08.911555 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:08.911688 (+ 133us) service_pool.cc:227] Handling call
0809 15:53:10.228715 (+1317027us) spinlock_profiling.cc:243] Waited 143 ms on lock 0x7b540055d7d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694edbc3 00007f796e43781a 00007f7967b2ad04 00007f7967b2ac92 00007f7967b2abc1 00007f7967b2ab3c 00007f7967b29df2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:10.228841 (+ 126us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":329096960}
I20260809 15:53:10.169615 14864 consensus_queue.cc:237] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 4.6, Last appended by leader: 6, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:10.172363 10922 raft_consensus.cc:2454] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
I20260809 15:53:10.264253 10848 raft_consensus.cc:2454] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:53:10.271253 10847 raft_consensus.cc:1261] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 6 index: 5. Preceding OpId from leader: term: 8 index: 6. (index mismatch)
I20260809 15:53:10.166787 14674 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:10.274163 14674 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:10.277637 14674 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:10.278867 14254 consensus_queue.cc:237] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 3.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:10.279312 14675 consensus_queue.cc:1035] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 15:53:10.280673 14753 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:10.281886 14858 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:10.207712 14612 raft_consensus.cc:686] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 8 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:10.172123 14697 consensus_queue.cc:237] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:10.172227 14847 consensus_queue.cc:237] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.5, Last appended by leader: 5, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:10.187925 14812 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:10.292076 14638 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:10.171479 14787 consensus_queue.cc:260] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.6, Last appended by leader: 6, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:10.206888 14775 raft_consensus.cc:3032] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 LEADER]: Stepping down as leader of term 7
I20260809 15:53:10.303027 14894 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:10.304330 14641 raft_consensus.cc:2726] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:10.267470 14760 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:10.322165 10847 raft_consensus.cc:1226] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:10.259406 10924 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 8.
I20260809 15:53:10.326260 10841 raft_consensus.cc:1261] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
I20260809 15:53:10.207309 14764 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:10.170750 14005 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:53:10.346679 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 213145us
I20260809 15:53:10.172053 14654 raft_consensus.cc:686] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [term 10 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
W20260809 15:53:10.356580 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 234979us
I20260809 15:53:10.171607 14561 raft_consensus.cc:504] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:10.377995 10978 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:10.377614 14764 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:10.378887 14787 raft_consensus.cc:3037] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:53:10.368345 14775 raft_consensus.cc:729] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:10.394640 14684 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:10.394472 14775 consensus_queue.cc:260] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 7.9, Last appended by leader: 9, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:10.384272 11606 leader_election.cc:302] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:10.385318 11015 raft_consensus.cc:1226] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:10.386054 10978 leader_election.cc:302] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:53:10.390484 14876 raft_consensus.cc:2741] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election decision vote started in defunct term 4: won
I20260809 15:53:10.401917 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:10.390581 14654 consensus_queue.cc:237] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 9, Committed index: 9, Last appended: 8.9, Last appended by leader: 9, Current term: 10, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:10.403450 11637 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41482 (request call id 840) took 1074 ms. Trace:
I20260809 15:53:10.402067 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:10.401835 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dd8bcd39f37d403f9d8d8f43d870a878" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:10.409202 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:10.410357 10847 raft_consensus.cc:2454] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 5.
I20260809 15:53:10.407644 10978 leader_election.cc:302] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:10.416118 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:10.416836 10922 raft_consensus.cc:2454] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 8.
I20260809 15:53:10.408075 11637 rpcz_store.cc:276] 0809 15:53:09.328809 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:09.329383 (+ 574us) service_pool.cc:227] Handling call
0809 15:53:09.329916 (+ 533us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:09.436532 (+106616us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:09.436575 (+ 43us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:09.453361 (+ 16786us) log.cc:842] Serialized 26 byte log entry
0809 15:53:09.765066 (+311705us) raft_consensus.cc:1631] Marking committed up to 7
0809 15:53:09.853068 (+ 88002us) log.cc:842] Serialized 14 byte log entry
0809 15:53:09.919704 (+ 66636us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:10.396929 (+477225us) spinlock_profiling.cc:243] Waited 34.6 ms on lock 0x7b54004cf754. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:10.396993 (+ 64us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:10.397061 (+ 68us) raft_consensus.cc:1671] finished
0809 15:53:10.403225 (+ 6164us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:10.403387 (+ 162us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":96304,"prepare.queue_time_us":238660,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":39,"raft.queue_time_us":150264,"raft.run_cpu_time_us":218,"raft.run_wall_time_us":18869,"spinlock_wait_cycles":79613824,"thread_start_us":370808,"threads_started":2,"wal-append.queue_time_us":341892}
I20260809 15:53:10.418906 10978 leader_election.cc:302] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:53:10.407070 11606 leader_election.cc:302] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:10.421797 11606 leader_election.cc:302] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:10.385972 14848 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:10.410880 14641 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:10.445310 14848 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:10.445866 14641 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:10.447338 14848 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:10.447424 11606 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:53:10.447892 14638 raft_consensus.cc:2726] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Leader pre-election lost for term 10. Reason: could not achieve majority
I20260809 15:53:10.448390 14857 consensus_queue.cc:1035] T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 15:53:10.412096 14910 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:10.451164 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:10.451140 14910 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:10.452013 14473 consensus_meta.cc:287] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.550s user 0.011s sys 0.000s
I20260809 15:53:10.453097 14868 raft_consensus.cc:2726] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:53:10.454039 10929 raft_consensus.cc:1226] T e80d8e2921024e68a1b7c56fda99ee80 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:10.454101 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:10.455075 14561 leader_election.cc:288] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:10.455688 14676 raft_consensus.cc:2726] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:53:10.455757 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e80d8e2921024e68a1b7c56fda99ee80" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:10.456113 10924 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:53:10.449740 11606 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:10.376837 14760 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:10.412252 14612 consensus_queue.cc:237] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 7.3, Last appended by leader: 3, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:10.444753 14894 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:10.446331 10838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:45578 (request call id 2147) took 1148 ms. Trace:
I20260809 15:53:10.475021 10838 rpcz_store.cc:276] 0809 15:53:09.297902 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:09.298022 (+ 120us) service_pool.cc:227] Handling call
0809 15:53:10.446157 (+1148135us) spinlock_profiling.cc:243] Waited 432 ms on lock 0x7b54001f04d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:10.446277 (+ 120us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":993315584}
I20260809 15:53:10.448639 14638 raft_consensus.cc:2781] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:53:10.457703 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:10.478719 10841 raft_consensus.cc:2454] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:53:10.442927 14812 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:10.403060 14005 raft_consensus.cc:2726] T 73ac44823c384ed99817c1ba3bba75ea P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:10.406206 14822 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:53:10.483330 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 331339us
I20260809 15:53:10.462191 11606 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:53:10.466511 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:10.456833 14784 raft_consensus.cc:2726] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:53:10.447904 14641 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:10.470156 14876 raft_consensus.cc:2781] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:10.412720 14407 raft_consensus.cc:2781] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 15:53:10.478039 14638 raft_consensus.cc:482] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:10.493006 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:10.477399 14861 raft_consensus.cc:2726] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 15:53:10.497414 11606 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:53:10.484169 10893 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:10.498539 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 14 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:10.498879 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 14 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:10.496042 14910 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:10.504226 14764 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:10.504231 10893 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:10.505817 10838 raft_consensus.cc:2396] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 14 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260809 15:53:10.506477 10929 raft_consensus.cc:2396] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 14 because replica has last-logged OpId of term: 10 index: 10, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260809 15:53:10.507267 10893 leader_election.cc:302] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:10.491765 14876 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:10.492010 14407 raft_consensus.cc:482] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:10.525612 14407 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:53:10.498281 14641 raft_consensus.cc:2781] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 15:53:10.514165 11606 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:10.512120 10893 leader_election.cc:302] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:10.515043 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c8eaf50bcb3457bb4b1e30406cc349f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:10.500155 14638 raft_consensus.cc:3037] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:10.508247 14876 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:10.508638 14861 raft_consensus.cc:2781] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:10.508387 14455 consensus_queue.cc:1046] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 3.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.008s
I20260809 15:53:10.512596 14832 raft_consensus.cc:2726] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:53:10.517230 14805 raft_consensus.cc:2781] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Leader election won for term 5
I20260809 15:53:10.518064 11637 raft_consensus.cc:1261] T 08f368b4f4e748c596049c81eaed3f70 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 7 index: 9. Preceding OpId from leader: term: 8 index: 10. (index mismatch)
I20260809 15:53:10.522164 11018 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2560) took 1118 ms. Trace:
I20260809 15:53:10.547344 11018 rpcz_store.cc:276] 0809 15:53:09.404176 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:09.508462 (+104286us) service_pool.cc:227] Handling call
0809 15:53:09.508714 (+ 252us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:09.795021 (+286307us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:09.795042 (+ 21us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:09.795067 (+ 25us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:53:09.795086 (+ 19us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:10.521915 (+726829us) spinlock_profiling.cc:243] Waited 286 ms on lock 0x7b54001a59d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:10.521962 (+ 47us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:10.522104 (+ 142us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":658084608}
I20260809 15:53:10.548626 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 14 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:10.549162 11018 raft_consensus.cc:2396] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 6 index: 8.
I20260809 15:53:10.524323 11633 raft_consensus.cc:1226] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 4. Current term is 5. Ops: [4.6-4.6]
I20260809 15:53:10.524585 14394 raft_consensus.cc:504] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:10.508499 14561 raft_consensus.cc:2781] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260809 15:53:10.551918 10893 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:10.530575 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:10.489228 14822 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:10.544458 14642 raft_consensus.cc:2726] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 13 FOLLOWER]: Leader pre-election lost for term 14. Reason: could not achieve majority
I20260809 15:53:10.556171 14764 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:10.558449 14805 raft_consensus.cc:686] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:10.558789 14764 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:10.559351 14805 consensus_queue.cc:237] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 2.5, Last appended by leader: 5, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:10.560631 14764 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:10.553066 14561 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:53:10.562707 14561 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Advancing to term 11
I20260809 15:53:10.532133 14473 raft_consensus.cc:3037] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:10.544631 14861 raft_consensus.cc:482] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:10.557725 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:10.529543 14641 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:10.562197 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:10.601326 10929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:10.608748 10843 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 8.
I20260809 15:53:10.610134 11606 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:53:10.615309 10847 raft_consensus.cc:1261] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 9 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
W20260809 15:53:10.615818 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 53977.8us
I20260809 15:53:10.616737 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b9fb9f88a6847af87fef710d1a2793b" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:10.617873 11637 raft_consensus.cc:1261] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 10 index: 8. (index mismatch)
W20260809 15:53:10.565685 14770 consensus_meta.cc:287] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.777s user 0.012s sys 0.000s
I20260809 15:53:10.624969 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:10.631062 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:10.658342 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5757bc9ac9441c3bdfa6c5b2b743b6d" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:10.628484 11017 raft_consensus.cc:2454] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:53:10.659708 10809 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:10.564534 14832 raft_consensus.cc:2726] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:53:10.600531 14861 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:10.611308 14641 raft_consensus.cc:3037] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Advancing to term 9
W20260809 15:53:10.669816 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 250983us
I20260809 15:53:10.611680 14760 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:10.725605 10978 leader_election.cc:302] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 4 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
W20260809 15:53:10.737897 10811 thread.cc:636] dns-resolver [worker] (thread pool) Time spent creating pthread: real 1.265s user 0.025s sys 0.037s
I20260809 15:53:10.719905 11604 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:10.737936 11632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 1058) took 1916 ms. Trace:
I20260809 15:53:10.667124 14845 raft_consensus.cc:2781] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:53:10.742755 11632 rpcz_store.cc:276] 0809 15:53:08.821291 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:08.821439 (+ 148us) service_pool.cc:227] Handling call
0809 15:53:08.831579 (+ 10140us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:09.074142 (+242563us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:09.074184 (+ 42us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:09.074254 (+ 70us) raft_consensus.cc:1631] Marking committed up to 7
0809 15:53:09.074301 (+ 47us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:10.737708 (+1663407us) spinlock_profiling.cc:243] Waited 242 ms on lock 0x7b540055d7d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:10.737758 (+ 50us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:10.737880 (+ 122us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":557259520}
I20260809 15:53:10.764961 11640 raft_consensus.cc:1226] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [term 8 LEADER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8. Ops: [7.7-7.7]
I20260809 15:53:10.668555 14812 raft_consensus.cc:2781] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 15:53:10.762334 10847 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:10.774319 10847 raft_consensus.cc:3037] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:53:10.776163 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54236 (request call id 2163) took 2300 ms. Trace:
I20260809 15:53:10.773528 10978 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:10.779913 10978 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 10 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:53:10.781236 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:10.776683 10931 rpcz_store.cc:276] 0809 15:53:08.475856 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:08.477235 (+ 1379us) service_pool.cc:227] Handling call
0809 15:53:08.477615 (+ 380us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:08.701481 (+223866us) raft_consensus.cc:1520] Early marking committed up to index 4
0809 15:53:08.717575 (+ 16094us) log.cc:842] Serialized 14 byte log entry
0809 15:53:08.781071 (+ 63496us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:08.781324 (+ 253us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:53:08.781466 (+ 142us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:09.423555 (+642089us) spinlock_profiling.cc:243] Waited 4.26 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:53:10.775930 (+1352375us) spinlock_profiling.cc:243] Waited 224 ms on lock 0x7b540018a954. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:10.775986 (+ 56us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:10.776109 (+ 123us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":22261,"prepare.run_cpu_time_us":39,"prepare.run_wall_time_us":40,"raft.queue_time_us":314,"raft.run_cpu_time_us":126,"raft.run_wall_time_us":19176,"spinlock_wait_cycles":524128256,"thread_start_us":62178,"threads_started":1,"wal-append.queue_time_us":201889,"wal-append.run_cpu_time_us":5007,"wal-append.run_wall_time_us":1674304}
I20260809 15:53:10.786520 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:10.787580 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:10.788730 11632 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 13.
I20260809 15:53:10.787153 11633 raft_consensus.cc:2454] T 6508f17228904310be3c95818973f3a1 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:53:10.773840 14845 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:10.763453 14348 raft_consensus.cc:2726] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:53:10.776464 14812 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:10.817294 11633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:10.773170 14917 raft_consensus.cc:2726] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election lost for term 4. Reason: could not achieve majority
W20260809 15:53:10.818912 10811 thread.cc:602] dns-resolver [worker] (thread pool) Time spent starting thread: real 1.346s user 0.026s sys 0.037s
I20260809 15:53:10.815191 11632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b4306bfa5464b0dbf8301756455ba50" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
W20260809 15:53:10.827627 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 1.35473e+06us
W20260809 15:53:10.828303 10811 leader_election.cc:334] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.818s (SENT)
I20260809 15:53:10.829066 10811 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:10.842453 10811 leader_election.cc:302] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:10.804104 14261 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:10.763900 14749 consensus_queue.cc:1035] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.062s
I20260809 15:53:10.764103 14784 consensus_queue.cc:1035] T 08f368b4f4e748c596049c81eaed3f70 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.063s
I20260809 15:53:10.827603 14787 raft_consensus.cc:2726] T 1faddf7ecc6e48ba9306d7ece1f936b1 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:10.854367 14612 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:10.874286 14809 raft_consensus.cc:2726] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority
I20260809 15:53:10.859107 10811 leader_election.cc:302] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:10.827178 14845 raft_consensus.cc:3037] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:10.887120 14612 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:10.839691 14812 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:53:10.896411 11018 raft_consensus.cc:1261] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260809 15:53:10.856143 14864 raft_consensus.cc:2781] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Leader pre-election won for term 10
I20260809 15:53:10.838806 14924 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:53:10.844256 14847 raft_consensus.cc:2726] T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:53:10.897280 10931 raft_consensus.cc:1261] T 0f891fe62df749108fa553cb0661a874 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 3 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260809 15:53:10.898926 14809 raft_consensus.cc:2726] T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260809 15:53:10.953714 10811 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
W20260809 15:53:10.897886 14930 log.cc:924] Time spent T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.106s user 0.000s sys 0.002s
I20260809 15:53:10.899603 14864 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:10.883201 14261 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:10.974852 10922 raft_consensus.cc:1261] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260809 15:53:10.973676 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:10.988561 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:10.989284 10840 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 13.
I20260809 15:53:10.904596 14394 leader_election.cc:288] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:10.991496 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
W20260809 15:53:10.994606 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 74404us
I20260809 15:53:10.996047 10811 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:11.038723 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:10.951711 14764 consensus_queue.cc:1035] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.041s
I20260809 15:53:10.966722 14822 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:11.041580 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:10.984730 14864 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Advancing to term 10
I20260809 15:53:11.066486 10924 raft_consensus.cc:595] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 8 LEADER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:11.067462 10893 leader_election.cc:302] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:11.072548 14394 raft_consensus.cc:2726] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:53:11.073546 11013 raft_consensus.cc:1226] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [term 8 LEADER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:11.076033 11606 leader_election.cc:302] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:11.075459 10893 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:10.978511 14918 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 13 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:11.085273 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:11.086793 10840 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 8.
I20260809 15:53:11.087807 11013 raft_consensus.cc:1261] T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260809 15:53:11.088573 10840 raft_consensus.cc:3037] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:11.091814 10838 raft_consensus.cc:1226] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:10.993501 14642 raft_consensus.cc:482] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:11.091907 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:11.042743 14648 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:11.034614 14760 raft_consensus.cc:2781] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260809 15:53:11.096743 10838 raft_consensus.cc:1261] T 369529a0023c4cf4a709bb1a480052b4 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260809 15:53:11.100594 11013 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Advancing to term 9
W20260809 15:53:11.063145 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 67211.4us
I20260809 15:53:11.062829 14784 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:11.062892 14247 raft_consensus.cc:2726] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:53:10.979220 14674 consensus_queue.cc:1035] T 1f9b7d9e57934596b377ee8dc79ea23e P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.105s
I20260809 15:53:11.107404 10811 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:11.092592 14822 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:11.147711 10893 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:53:11.170940 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 63589.9us
I20260809 15:53:11.038895 14868 consensus_queue.cc:1035] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.068s
I20260809 15:53:11.160967 11013 raft_consensus.cc:2454] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 9.
I20260809 15:53:11.104771 14760 raft_consensus.cc:482] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260809 15:53:11.150902 14876 consensus_meta.cc:287] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.598s user 0.006s sys 0.004s
I20260809 15:53:10.958482 14924 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.097172 14487 consensus_queue.cc:1035] T 0f891fe62df749108fa553cb0661a874 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.065s
I20260809 15:53:11.182386 10840 raft_consensus.cc:1261] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 6 index: 7. Preceding OpId from leader: term: 7 index: 8. (index mismatch)
I20260809 15:53:11.171672 10811 leader_election.cc:302] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:11.098966 14402 consensus_queue.cc:1035] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.071s
I20260809 15:53:11.197088 10840 raft_consensus.cc:1261] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 8 index: 7. (index mismatch)
I20260809 15:53:11.156415 14827 raft_consensus.cc:2726] T 220f58b662044f089f2be802ed24284a P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:53:11.193706 10811 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:11.198969 10811 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:53:11.196138 14932 log.cc:924] Time spent T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.312s user 0.001s sys 0.000s
I20260809 15:53:11.101830 14642 raft_consensus.cc:504] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:11.185678 14925 log.cc:924] Time spent T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.371s user 0.001s sys 0.000s
I20260809 15:53:11.184139 14916 raft_consensus.cc:2726] T 02f25d93293a40729fd751a19ab9adeb P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:11.101449 14918 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 13 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.204339 10840 raft_consensus.cc:1261] T cfa61e868cfb4915b0572f8e33745f43 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 8 index: 9. (index mismatch)
I20260809 15:53:11.185972 14760 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Advancing to term 11
I20260809 15:53:11.201829 10811 leader_election.cc:302] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters:
I20260809 15:53:11.146358 14648 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.167634 14784 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:11.200798 14919 raft_consensus.cc:2726] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260809 15:53:11.219231 10847 raft_consensus.cc:2454] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 5.
I20260809 15:53:11.196595 14247 raft_consensus.cc:2726] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 15:53:11.196475 14809 raft_consensus.cc:2726] T 241f07ed14b041cdb37e8ec792283d52 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:11.243158 10838 raft_consensus.cc:1226] T 2b4306bfa5464b0dbf8301756455ba50 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 4. Current term is 5. Ops: []
I20260809 15:53:11.204195 14785 raft_consensus.cc:2726] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 13 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority
W20260809 15:53:11.250269 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 470413us
I20260809 15:53:11.203240 14847 raft_consensus.cc:2726] T 8777c94cbe134bf3aa222691d76ba470 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:53:11.176600 14822 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.251814 10978 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:11.207283 14612 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:11.253422 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 106070us
I20260809 15:53:11.195747 14876 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.242300 14801 raft_consensus.cc:3032] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:53:11.279757 11634 raft_consensus.cc:1261] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260809 15:53:11.280718 10847 raft_consensus.cc:1226] T 049fe6a2ea4f4a15a98b9c6c5755173d P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8. Ops: [5.6-7.7]
I20260809 15:53:11.209438 14521 consensus_queue.cc:1035] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.046s
I20260809 15:53:11.284825 11634 raft_consensus.cc:1226] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260809 15:53:11.284778 10847 raft_consensus.cc:1261] T bf6504ce1c0d4241bfb97eb553c8df35 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260809 15:53:11.288414 11633 raft_consensus.cc:1261] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 7 index: 8. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260809 15:53:11.289585 11634 raft_consensus.cc:1261] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 2 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260809 15:53:11.272943 14247 raft_consensus.cc:2781] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:53:11.291504 10922 raft_consensus.cc:3032] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 15:53:11.269655 14947 raft_consensus.cc:482] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:11.254666 14894 consensus_queue.cc:1046] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 5.5, Next index: 6, Last known committed idx: 5, Time since last communication: 0.031s
I20260809 15:53:11.292853 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "898f053882ab4598a37b00ed268ca6b3" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:11.303220 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:11.294472 10922 raft_consensus.cc:729] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:11.253156 14455 consensus_queue.cc:1046] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.038s
I20260809 15:53:11.295662 11634 raft_consensus.cc:1261] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 9. Preceding OpId from leader: term: 10 index: 10. (index mismatch)
I20260809 15:53:11.276897 14917 raft_consensus.cc:2781] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 15:53:11.311805 10978 leader_election.cc:302] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
W20260809 15:53:11.291049 14944 log.cc:924] Time spent T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.313s user 0.000s sys 0.001s
I20260809 15:53:11.283985 14801 raft_consensus.cc:729] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 5 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
W20260809 15:53:11.294284 14942 log.cc:924] Time spent T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.255s user 0.000s sys 0.001s
I20260809 15:53:11.317516 10891 leader_election.cc:302] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:11.304584 10922 consensus_queue.cc:260] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.319478 10922 raft_consensus.cc:3037] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
W20260809 15:53:11.319624 14928 log.cc:924] Time spent T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.162s user 0.000s sys 0.001s
I20260809 15:53:11.320474 14784 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:11.321089 14784 consensus_queue.cc:1035] T 241f07ed14b041cdb37e8ec792283d52 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 15:53:11.294966 14247 raft_consensus.cc:482] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:11.323678 14247 raft_consensus.cc:3037] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:11.320492 14822 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:11.315328 14917 raft_consensus.cc:482] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
W20260809 15:53:11.329679 14937 log.cc:924] Time spent T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.233s user 0.001s sys 0.000s
W20260809 15:53:11.332541 14950 log.cc:924] Time spent T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.063s user 0.000s sys 0.001s
I20260809 15:53:11.304812 14947 raft_consensus.cc:504] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.321110 14801 consensus_queue.cc:260] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.306728 14612 consensus_queue.cc:1035] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.021s
I20260809 15:53:11.272149 14948 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.046s
I20260809 15:53:11.375257 11642 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 1076) took 1274 ms. Trace:
I20260809 15:53:11.477655 11642 rpcz_store.cc:276] 0809 15:53:10.100435 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:10.313989 (+213554us) service_pool.cc:227] Handling call
0809 15:53:10.315738 (+ 1749us) raft_consensus.cc:1457] Updating replica for 3 ops
0809 15:53:10.316213 (+ 475us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:10.316268 (+ 55us) raft_consensus.cc:1525] Triggering prepare for 3 ops
0809 15:53:10.317021 (+ 753us) log.cc:842] Serialized 78 byte log entry
0809 15:53:10.661491 (+344470us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:53:10.732567 (+ 71076us) log.cc:842] Serialized 14 byte log entry
0809 15:53:10.732886 (+ 319us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:10.732983 (+ 97us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:11.349869 (+616886us) raft_consensus.cc:1671] finished
0809 15:53:11.349891 (+ 22us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:11.375186 (+ 25295us) inbound_call.cc:173] Queueing success response
Metrics: {"prepare.queue_time_us":72133,"prepare.run_cpu_time_us":94,"prepare.run_wall_time_us":5149,"raft.queue_time_us":6887,"raft.run_cpu_time_us":152,"raft.run_wall_time_us":24565,"spinlock_wait_cycles":9203584,"thread_start_us":344039,"threads_started":1,"wal-append.queue_time_us":721572}
I20260809 15:53:11.376296 14666 raft_consensus.cc:482] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:11.381434 14405 raft_consensus.cc:2781] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader election won for term 5
W20260809 15:53:11.377357 14940 log.cc:924] Time spent T e5757bc9ac9441c3bdfa6c5b2b743b6d P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.271s user 0.002s sys 0.001s
W20260809 15:53:11.381572 10891 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 64152.1us
I20260809 15:53:11.395848 10922 raft_consensus.cc:1261] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 7. (term mismatch)
I20260809 15:53:11.295892 14261 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
W20260809 15:53:11.462548 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 260798us
W20260809 15:53:11.480729 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 266388us
I20260809 15:53:11.481218 11015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 847) took 1540 ms. Trace:
I20260809 15:53:11.496457 11015 rpcz_store.cc:276] 0809 15:53:09.940947 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:09.942699 (+ 1752us) service_pool.cc:227] Handling call
0809 15:53:09.942968 (+ 269us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:10.398995 (+456027us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:11.480994 (+1081999us) spinlock_profiling.cc:243] Waited 442 ms on lock 0x7b54002da854. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:11.481162 (+ 168us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1016836992}
W20260809 15:53:11.462056 14909 log.cc:924] Time spent T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.207s user 0.003s sys 0.003s
W20260809 15:53:11.483543 11010 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:50542 (request call id 2557) took 2084 ms (client timeout 2103 ms). Trace:
W20260809 15:53:11.498714 11010 rpcz_store.cc:269] 0809 15:53:09.394560 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:09.460295 (+ 65735us) service_pool.cc:227] Handling call
0809 15:53:11.478446 (+2018151us) spinlock_profiling.cc:243] Waited 173 ms on lock 0x7b54003f8c54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:11.478615 (+ 169us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":398241664}
I20260809 15:53:11.376839 14960 raft_consensus.cc:3032] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:53:11.375607 14916 raft_consensus.cc:482] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:53:11.461342 14956 log.cc:924] Time spent T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.208s user 0.000s sys 0.003s
W20260809 15:53:11.480232 14939 log.cc:924] Time spent T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.310s user 0.000s sys 0.004s
W20260809 15:53:11.464815 14473 consensus_meta.cc:287] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.900s user 0.011s sys 0.002s
I20260809 15:53:11.384768 14949 consensus_queue.cc:1046] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 7.3, Next index: 4, Last known committed idx: 2, Time since last communication: 0.115s
I20260809 15:53:11.507786 14473 raft_consensus.cc:2726] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:11.507503 14916 raft_consensus.cc:504] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.385672 14917 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Advancing to term 9
W20260809 15:53:11.509034 14953 log.cc:924] Time spent T 08f368b4f4e748c596049c81eaed3f70 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.155s user 0.000s sys 0.000s
W20260809 15:53:11.509650 14684 consensus_meta.cc:287] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 1.114s user 0.010s sys 0.003s
I20260809 15:53:11.501144 11013 raft_consensus.cc:1261] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 4 index: 6. Preceding OpId from leader: term: 5 index: 6. (term mismatch)
I20260809 15:53:11.528510 11013 pending_rounds.cc:76] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa: Aborting all ops after (but not including) 5
I20260809 15:53:11.528803 11013 pending_rounds.cc:97] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa: Aborting uncommitted NO_OP operation due to leader change: 4.6
I20260809 15:53:11.529065 11013 raft_consensus.cc:2866] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260809 15:53:11.464615 14561 consensus_meta.cc:287] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.899s user 0.005s sys 0.010s
I20260809 15:53:11.481050 14832 raft_consensus.cc:2781] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 9 FOLLOWER]: Leader election won for term 9
I20260809 15:53:11.504129 14405 raft_consensus.cc:686] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:11.501964 14960 raft_consensus.cc:729] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:11.510250 14962 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:11.500514 14924 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:11.512002 14916 leader_election.cc:288] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:11.513394 10841 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 902) took 1020 ms. Trace:
I20260809 15:53:11.543538 10841 rpcz_store.cc:276] 0809 15:53:10.492983 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:10.494271 (+ 1288us) service_pool.cc:227] Handling call
0809 15:53:10.494650 (+ 379us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:10.495053 (+ 403us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:10.495092 (+ 39us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:10.495581 (+ 489us) log.cc:842] Serialized 26 byte log entry
0809 15:53:10.764039 (+268458us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:53:10.764080 (+ 41us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:10.764147 (+ 67us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:11.513188 (+749041us) raft_consensus.cc:1671] finished
0809 15:53:11.513209 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:11.513338 (+ 129us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":1050,"raft.run_cpu_time_us":109,"raft.run_wall_time_us":6398,"thread_start_us":254880,"threads_started":1,"wal-append.queue_time_us":492932}
I20260809 15:53:11.494972 10922 pending_rounds.cc:76] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3: Aborting all ops after (but not including) 6
I20260809 15:53:11.545812 10922 pending_rounds.cc:97] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3: Aborting uncommitted NO_OP operation due to leader change: 7.7
I20260809 15:53:11.546171 10922 raft_consensus.cc:2866] T 049fe6a2ea4f4a15a98b9c6c5755173d P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260809 15:53:11.546324 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:11.546617 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 14 candidate_status { last_received { term: 12 index: 11 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:11.514741 10837 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 891) took 2143 ms. Trace:
I20260809 15:53:11.526312 14949 consensus_queue.cc:1225] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer d74944fa9fa14ec78b0640ea709ff3aa log is divergent from this leader: its last log entry 8.4 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 2
I20260809 15:53:11.527915 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:11.498535 14801 raft_consensus.cc:3037] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:11.531919 10929 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 8.
W20260809 15:53:11.538734 14861 consensus_meta.cc:287] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.801s user 0.009s sys 0.002s
I20260809 15:53:11.547149 10837 rpcz_store.cc:276] 0809 15:53:09.371219 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:09.809404 (+438185us) service_pool.cc:227] Handling call
0809 15:53:09.809886 (+ 482us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:09.826886 (+ 17000us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:09.826915 (+ 29us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:09.838041 (+ 11126us) log.cc:842] Serialized 26 byte log entry
0809 15:53:10.032815 (+194774us) raft_consensus.cc:1631] Marking committed up to 8
0809 15:53:10.242330 (+209515us) log.cc:842] Serialized 14 byte log entry
0809 15:53:10.242598 (+ 268us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:10.242689 (+ 91us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:10.525487 (+282798us) spinlock_profiling.cc:243] Waited 23.9 ms on lock 0x7b180026fbc0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f79665cb638 00007f79665ceb11 00007f79665ceac9 00007f79665cea59 00007f79665cea21 00007f79665cdd1d 00007f796f82f204 00007f796f82f039 00007f79665addc6
0809 15:53:11.209213 (+683726us) spinlock_profiling.cc:243] Waited 19.7 ms on lock 0x7b180026fbc0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f79665cb638 00007f79665ceb11 00007f79665ceac9 00007f79665cea59 00007f79665cea21 00007f79665cdd1d 00007f796f82f204 00007f796f82f039 00007f79665addc6
0809 15:53:11.514511 (+305298us) raft_consensus.cc:1671] finished
0809 15:53:11.514532 (+ 21us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:11.514673 (+ 141us) inbound_call.cc:173] Queueing success response
Metrics: {"mutex_wait_us":18107,"prepare.queue_time_us":211552,"prepare.run_cpu_time_us":67,"prepare.run_wall_time_us":5779,"raft.queue_time_us":17129,"raft.run_cpu_time_us":206,"raft.run_wall_time_us":1460,"spinlock_wait_cycles":100422912,"thread_start_us":276098,"threads_started":2,"wal-append.queue_time_us":374500}
W20260809 15:53:11.552891 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 54543.2us
I20260809 15:53:11.492208 14666 raft_consensus.cc:504] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:53:11.544762 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 231697us
I20260809 15:53:11.518543 11018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:11.547220 10921 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 13.
I20260809 15:53:11.486181 14951 consensus_queue.cc:1035] T 220f58b662044f089f2be802ed24284a P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.057s
W20260809 15:53:11.535913 14943 log.cc:924] Time spent T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.313s user 0.001s sys 0.000s
I20260809 15:53:11.550678 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:53:11.533291 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 260513us
I20260809 15:53:11.533336 14642 leader_election.cc:288] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:11.549038 14962 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.558573 14917 raft_consensus.cc:504] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:53:11.560258 14812 consensus_meta.cc:287] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.608s user 0.008s sys 0.003s
W20260809 15:53:11.560596 14638 consensus_meta.cc:287] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 1.013s user 0.010s sys 0.002s
I20260809 15:53:11.536130 14561 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.562436 14876 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:11.563014 14969 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:11.600127 14947 leader_election.cc:288] T cfa61e868cfb4915b0572f8e33745f43 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:11.600381 14969 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:11.581781 14960 consensus_queue.cc:260] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 6.6, Last appended by leader: 6, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:11.602613 14969 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:11.602923 14917 leader_election.cc:288] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:11.603233 14960 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
W20260809 15:53:11.604811 14864 consensus_meta.cc:287] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 0.505s user 0.011s sys 0.000s
I20260809 15:53:11.605239 14864 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:11.605639 14492 raft_consensus.cc:3037] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:11.581660 14966 consensus_queue.cc:1035] T dd8bcd39f37d403f9d8d8f43d870a878 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.028s
I20260809 15:53:11.607756 14962 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:11.564289 10924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.LeaderStepDown from 127.0.0.1:34958 (request call id 41) took 1120 ms. Trace:
I20260809 15:53:11.608579 10924 rpcz_store.cc:276] 0809 15:53:10.443797 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:10.455989 (+ 12192us) service_pool.cc:227] Handling call
0809 15:53:11.564102 (+1108113us) spinlock_profiling.cc:243] Waited 610 ms on lock 0x7b54003ee254. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694edbc3 00007f796e43781a 00007f7967b2ad04 00007f7967b2ac92 00007f7967b2abc1 00007f7967b2ab3c 00007f7967b29df2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:11.564195 (+ 93us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1402357632}
I20260809 15:53:11.568495 10978 leader_election.cc:302] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:11.610301 10978 leader_election.cc:302] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
W20260809 15:53:11.611121 14845 consensus_meta.cc:287] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.711s user 0.011s sys 0.000s
I20260809 15:53:11.611490 14845 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.611882 10978 leader_election.cc:302] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:53:11.612922 10978 leader_election.cc:302] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:11.571740 10893 leader_election.cc:302] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:11.574275 14254 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:11.613441 14845 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Requested vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:11.613849 14254 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.614681 14970 raft_consensus.cc:2726] T 7c8eaf50bcb3457bb4b1e30406cc349f P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election lost for term 6. Reason: could not achieve majority
I20260809 15:53:11.615337 14970 raft_consensus.cc:2781] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:53:11.615675 14970 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:11.615935 14254 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:53:11.576925 11635 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41478 (request call id 1092) took 2206 ms (client timeout 1703 ms). Trace:
W20260809 15:53:11.617018 11635 rpcz_store.cc:269] 0809 15:53:09.345198 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:09.363278 (+ 18080us) service_pool.cc:227] Handling call
0809 15:53:11.551698 (+2188420us) spinlock_profiling.cc:243] Waited 175 ms on lock 0x7b54004d3fd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:11.551790 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":403629312}
I20260809 15:53:11.541442 14832 raft_consensus.cc:686] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 9 LEADER]: Becoming Leader. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:11.618566 14832 consensus_queue.cc:237] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:11.619572 11013 raft_consensus.cc:1226] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 8. Current term is 9. Ops: []
W20260809 15:53:11.621012 10848 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 881) took 2942 ms (client timeout 3373 ms). Trace:
I20260809 15:53:11.549948 14851 raft_consensus.cc:482] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:11.621647 14847 raft_consensus.cc:482] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:11.622004 14851 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.622202 14847 raft_consensus.cc:504] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:11.560711 14896 log.cc:924] Time spent T e6e09d263f96440a99c99d5a5c6ea477 P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 1.794s user 0.002s sys 0.002s
I20260809 15:53:11.593660 10921 raft_consensus.cc:1261] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260809 15:53:11.624135 14847 leader_election.cc:288] T 8feeb57bf5cf4fd1bb17ab7f575222d0 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:11.624892 10839 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2395) took 4200 ms. Trace:
I20260809 15:53:11.625250 10839 rpcz_store.cc:276] 0809 15:53:07.424024 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:07.424866 (+ 842us) service_pool.cc:227] Handling call
0809 15:53:07.430756 (+ 5890us) raft_consensus.cc:1457] Updating replica for 2 ops
0809 15:53:07.431104 (+ 348us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:07.431138 (+ 34us) raft_consensus.cc:1525] Triggering prepare for 2 ops
0809 15:53:09.412648 (+1981510us) spinlock_profiling.cc:243] Waited 9.96 ms on lock 0x7b2c000120e8. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7967ed6e11 00007f7967ee6442 00007f796952ec11 00007f79694f358a 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772
0809 15:53:09.413283 (+ 635us) log.cc:842] Serialized 52 byte log entry
0809 15:53:09.572926 (+159643us) raft_consensus.cc:1631] Marking committed up to 7
0809 15:53:09.572962 (+ 36us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:09.573035 (+ 73us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:11.559999 (+1986964us) spinlock_profiling.cc:243] Waited 10.9 ms on lock 0x7b44000232cc. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796640ff11 00007f79664dda95 00007f79664de4b2 00007f7969186511 00007f7969181e1a 00007f7969175bee 00007f7969174f36 00007f7969174b58 00007f796917e551 00007f796917e509 00007f796917e499
0809 15:53:11.624658 (+ 64659us) raft_consensus.cc:1671] finished
0809 15:53:11.624681 (+ 23us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:11.624814 (+ 133us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":353,"raft.run_cpu_time_us":181,"raft.run_wall_time_us":34054,"spinlock_wait_cycles":47986816,"thread_start_us":159216,"threads_started":1,"wal-append.queue_time_us":237292}
I20260809 15:53:11.595155 14638 raft_consensus.cc:504] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.551347 14924 consensus_queue.cc:1046] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.042s
I20260809 15:53:11.578012 14861 raft_consensus.cc:504] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.627396 14877 raft_consensus.cc:3032] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 15:53:11.627748 14877 raft_consensus.cc:729] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:53:11.628259 14638 leader_election.cc:288] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:11.592830 14642 raft_consensus.cc:482] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:11.628684 14861 leader_election.cc:288] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:11.628546 14877 consensus_queue.cc:260] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.628981 14642 raft_consensus.cc:504] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 6 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.631609 14642 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:11.599213 14348 consensus_queue.cc:1035] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 15:53:11.614238 14612 raft_consensus.cc:2726] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:11.616024 14970 raft_consensus.cc:3037] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
W20260809 15:53:11.621364 10848 rpcz_store.cc:269] 0809 15:53:08.678262 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:08.679133 (+ 871us) service_pool.cc:227] Handling call
0809 15:53:11.620845 (+2941712us) spinlock_profiling.cc:243] Waited 1.56 s on lock 0x7b540001d454. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:11.620937 (+ 92us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3585567104}
I20260809 15:53:11.634068 11632 raft_consensus.cc:1226] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:11.598150 14918 leader_election.cc:288] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 14 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:11.598729 14967 consensus_queue.cc:1035] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 5, Time since last communication: 0.008s
I20260809 15:53:11.631542 14966 consensus_queue.cc:1046] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
W20260809 15:53:11.646556 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 121762us
I20260809 15:53:11.646948 10978 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:11.647786 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:53:11.651229 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 79569.2us
I20260809 15:53:11.655469 10843 raft_consensus.cc:1261] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260809 15:53:11.659505 10839 raft_consensus.cc:1261] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 5 index: 6. (index mismatch)
I20260809 15:53:11.657578 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:11.660620 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 6 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:11.661147 10920 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:11.661640 10841 raft_consensus.cc:1261] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 5 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260809 15:53:11.661986 10978 leader_election.cc:302] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:53:11.663477 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:11.663618 10978 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:11.664037 11017 raft_consensus.cc:3037] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:11.665185 10927 raft_consensus.cc:1261] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 8 index: 5. Preceding OpId from leader: term: 10 index: 6. (index mismatch)
I20260809 15:53:11.666755 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:11.668428 11012 raft_consensus.cc:2396] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 13 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 20730d75ad954a82b126148f8152a9f3 for term 14 because replica has last-logged OpId of term: 12 index: 11, which is greater than that of the candidate, which has last-logged OpId of term: 10 index: 10.
I20260809 15:53:11.668995 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:11.669292 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:11.669557 10839 raft_consensus.cc:3037] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:53:11.617990 14753 raft_consensus.cc:2781] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election won for term 8
I20260809 15:53:11.631551 14876 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:11.676630 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:11.677241 10840 raft_consensus.cc:2454] T 855d87c571d0427ba84900052dd9e2c0 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
W20260809 15:53:11.662719 14407 consensus_meta.cc:287] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa: Time spent flushing consensus metadata: real 1.137s user 0.008s sys 0.004s
W20260809 15:53:11.679360 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 83953.5us
I20260809 15:53:11.614022 14845 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:11.680996 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:11.684073 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:11.684763 11012 raft_consensus.cc:2454] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:53:11.684890 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:11.639868 14648 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:11.686275 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 11 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:11.668538 14675 raft_consensus.cc:2726] T 0b9fb9f88a6847af87fef710d1a2793b P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:53:11.686978 10920 raft_consensus.cc:2396] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 for term 7 because replica has last-logged OpId of term: 6 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 5 index: 5.
I20260809 15:53:11.689860 11012 raft_consensus.cc:1226] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8. Ops: []
W20260809 15:53:11.681447 14866 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 1.559s user 0.009s sys 0.007s
I20260809 15:53:11.651713 14951 raft_consensus.cc:482] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:11.596495 14812 raft_consensus.cc:504] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.677983 14917 raft_consensus.cc:2781] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:53:11.698762 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1faddf7ecc6e48ba9306d7ece1f936b1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:11.675776 14612 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260809 15:53:11.702134 11017 raft_consensus.cc:2454] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 7.
W20260809 15:53:11.705001 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 45109.7us
I20260809 15:53:11.703001 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:11.717391 10848 raft_consensus.cc:2379] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 8: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:53:11.717311 10893 leader_election.cc:302] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:11.718889 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:11.722954 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:11.730685 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "369529a0023c4cf4a709bb1a480052b4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:11.731555 10924 raft_consensus.cc:1226] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [term 9 LEADER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 8. Current term is 9. Ops: []
I20260809 15:53:11.733196 11012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:11.734525 11635 raft_consensus.cc:1261] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 7 index: 7. Preceding OpId from leader: term: 8 index: 8. (index mismatch)
I20260809 15:53:11.735047 10811 leader_election.cc:302] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:11.736367 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:53:11.736933 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 66475.7us
I20260809 15:53:11.736929 10839 raft_consensus.cc:2454] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 9.
I20260809 15:53:11.737426 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:11.737128 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:11.741976 10930 raft_consensus.cc:2379] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 11: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:11.742843 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:11.741559 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6d4d6dee78914014adab40a391b02955" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:11.738525 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:11.745327 10839 raft_consensus.cc:3037] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:11.737818 10891 leader_election.cc:302] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536
I20260809 15:53:11.752357 11604 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:11.758989 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:11.761456 10809 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:11.763136 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:11.764218 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:11.764815 11642 raft_consensus.cc:2454] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 6.
W20260809 15:53:11.700668 14866 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 1.583s user 0.010s sys 0.008s
I20260809 15:53:11.766727 11637 raft_consensus.cc:1226] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 5. Current term is 6. Ops: []
I20260809 15:53:11.764554 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:11.768596 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:11.744817 10922 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 8.
I20260809 15:53:11.710206 14960 raft_consensus.cc:2726] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 15:53:11.769194 11642 raft_consensus.cc:2454] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 5.
I20260809 15:53:11.710678 14917 raft_consensus.cc:482] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:11.659269 14405 consensus_queue.cc:237] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:11.772850 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 55651.3us
I20260809 15:53:11.686833 14407 raft_consensus.cc:504] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:11.775466 11606 leader_election.cc:302] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:11.682255 14876 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.777426 10839 raft_consensus.cc:2454] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:53:11.781915 11637 raft_consensus.cc:1226] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 5. Current term is 6. Ops: []
I20260809 15:53:11.781888 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:11.776073 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "241f07ed14b041cdb37e8ec792283d52" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:11.783068 10839 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 10.
I20260809 15:53:11.778739 11604 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 4 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:11.753892 14730 raft_consensus.cc:2726] T e5757bc9ac9441c3bdfa6c5b2b743b6d P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:11.785254 10922 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:53:11.785889 10922 raft_consensus.cc:595] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:11.786243 10922 raft_consensus.cc:603] T 9602fc7c9e814af09c87e4cb17fdb08d P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 15:53:11.697947 14845 raft_consensus.cc:504] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } attrs { replace: true } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.781064 11606 leader_election.cc:302] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:11.766209 14785 raft_consensus.cc:2781] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:53:11.733793 14770 consensus_queue.cc:1035] T 9d3f35b0ae8e4b02b72f832ca1518464 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.034s
I20260809 15:53:11.801789 11606 leader_election.cc:302] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:11.802467 10811 leader_election.cc:302] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 4 responses out of 4 voters: 2 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
W20260809 15:53:11.804742 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 57777.3us
I20260809 15:53:11.808902 11606 leader_election.cc:302] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:11.811278 11606 leader_election.cc:302] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:11.826906 11011 raft_consensus.cc:1226] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 8. Current term is 9. Ops: []
I20260809 15:53:11.827301 11017 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 8.
I20260809 15:53:11.773022 14861 raft_consensus.cc:2781] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:53:11.735065 14247 raft_consensus.cc:504] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:11.785195 14760 consensus_meta.cc:287] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.537s user 0.009s sys 0.001s
I20260809 15:53:11.773104 14948 raft_consensus.cc:2781] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election won for term 7
I20260809 15:53:11.832722 10841 raft_consensus.cc:2454] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 7.
I20260809 15:53:11.781054 14917 raft_consensus.cc:3037] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:11.773128 14966 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:11.798218 14638 raft_consensus.cc:2781] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Leader pre-election won for term 6
I20260809 15:53:11.804982 14642 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Advancing to term 11
I20260809 15:53:11.828583 14924 raft_consensus.cc:2726] T 1faddf7ecc6e48ba9306d7ece1f936b1 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:11.826720 14983 raft_consensus.cc:2781] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 15:53:11.780001 14981 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:11.776928 14348 consensus_queue.cc:1035] T 049fe6a2ea4f4a15a98b9c6c5755173d P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.061s
I20260809 15:53:11.826709 14254 raft_consensus.cc:2726] T 369529a0023c4cf4a709bb1a480052b4 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election lost for term 9. Reason: could not achieve majority
I20260809 15:53:11.831986 14868 raft_consensus.cc:2726] T 241f07ed14b041cdb37e8ec792283d52 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader election lost for term 8. Reason: could not achieve majority
I20260809 15:53:11.952945 14561 leader_election.cc:288] T 1f9b7d9e57934596b377ee8dc79ea23e P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 11 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:11.955821 14775 raft_consensus.cc:3037] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:11.831707 14962 raft_consensus.cc:2781] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader election won for term 6
I20260809 15:53:11.719029 14951 raft_consensus.cc:504] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 3 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } attrs { replace: true } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.958911 14962 raft_consensus.cc:686] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [term 6 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
W20260809 15:53:11.960755 14954 log.cc:924] Time spent T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.667s user 0.008s sys 0.024s
I20260809 15:53:11.960379 14962 consensus_queue.cc:237] T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 4.5, Last appended by leader: 5, Current term: 6, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.798532 14455 consensus_queue.cc:1046] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 7.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.114s
I20260809 15:53:11.963112 14455 raft_consensus.cc:3032] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 15:53:11.963565 14455 raft_consensus.cc:729] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:11.964226 11640 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 1071) took 2504 ms. Trace:
I20260809 15:53:11.964994 14951 leader_election.cc:288] T 8975b3e7ddc544bc888dd61fffc5313b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:11.964637 11640 rpcz_store.cc:276] 0809 15:53:09.459832 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:09.980553 (+520721us) service_pool.cc:227] Handling call
0809 15:53:09.987084 (+ 6531us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:10.773458 (+786374us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:11.963931 (+1190473us) spinlock_profiling.cc:243] Waited 777 ms on lock 0x7b540008a054. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:11.964166 (+ 235us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1788220800}
W20260809 15:53:11.940749 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 73202.6us
I20260809 15:53:11.792464 14918 consensus_queue.cc:1035] T 908409e5747b470cabbc8aed443397f7 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.120s
I20260809 15:53:11.779872 14832 consensus_queue.cc:1035] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.081s
W20260809 15:53:11.926313 14641 consensus_meta.cc:287] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 1.183s user 0.013s sys 0.000s
W20260809 15:53:11.946923 14975 log.cc:924] Time spent T e42f46986036495ab1735d596ab06d6d P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.246s user 0.001s sys 0.000s
I20260809 15:53:11.881253 14864 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 10 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:11.881880 14733 raft_consensus.cc:2726] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260809 15:53:11.877079 14638 raft_consensus.cc:482] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:11.860332 14666 leader_election.cc:288] T 3d87bec6301b44c7b8dd46364e6110a8 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:11.835999 14805 raft_consensus.cc:482] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:53:11.805141 14973 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 9, Time since last communication: 0.166s
I20260809 15:53:11.870872 14760 raft_consensus.cc:504] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 11 FOLLOWER]: Starting leader election with config: opid_index: 7 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.832829 14861 raft_consensus.cc:482] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:11.828439 14847 consensus_queue.cc:1035] T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.110s
I20260809 15:53:11.911821 14966 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.020133 14638 raft_consensus.cc:3037] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:11.861707 14948 raft_consensus.cc:482] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting leader election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
W20260809 15:53:12.018354 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 210608us
W20260809 15:53:12.068094 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 276267us
I20260809 15:53:12.076062 10922 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:12.081352 10840 raft_consensus.cc:2379] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 11 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 11: Already voted for candidate a53441872c0c416b8e2f26f52399f536 in this term.
I20260809 15:53:12.084492 10978 leader_election.cc:302] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:12.086468 10978 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:12.084471 10843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:12.038069 14861 raft_consensus.cc:3037] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:12.087677 10843 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P a53441872c0c416b8e2f26f52399f536 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 13.
I20260809 15:53:12.011008 14641 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:11.964656 14455 consensus_queue.cc:260] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.025732 14864 consensus_queue.cc:1035] T 9a5ecb5bb6e347a6921a7c894853e92e P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 2, Time since last communication: 0.063s
I20260809 15:53:12.069608 14812 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:12.077335 14948 raft_consensus.cc:3037] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:12.020213 14947 consensus_queue.cc:1035] T c695a0d40c7e4b04a1f7a1113954a8e4 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.063s
I20260809 15:53:12.111024 11634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9"
I20260809 15:53:12.112692 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "11cab658217a44bebba11963c723e3dc" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:12.115449 10848 raft_consensus.cc:3037] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:53:12.112375 11640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b2b0df042e44725a7c91deae3df1956" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 14 candidate_status { last_received { term: 10 index: 10 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:12.116612 11640 raft_consensus.cc:2454] T 0b2b0df042e44725a7c91deae3df1956 P c7aa762318cb4e97875333f10679a9f9 [term 13 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 13.
I20260809 15:53:11.970009 14981 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.088595 14753 raft_consensus.cc:686] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:12.034799 14805 raft_consensus.cc:504] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:12.124048 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:12.132046 10922 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "a53441872c0c416b8e2f26f52399f536"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:53:12.132787 10922 raft_consensus.cc:595] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Received request to transfer leadership to TS a53441872c0c416b8e2f26f52399f536
I20260809 15:53:12.133307 10922 raft_consensus.cc:603] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Rejecting request to transer leadership while not leader
W20260809 15:53:12.133126 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 46772.9us
I20260809 15:53:12.142439 10848 raft_consensus.cc:2454] T 11cab658217a44bebba11963c723e3dc P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 9.
I20260809 15:53:12.143479 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "908409e5747b470cabbc8aed443397f7" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:12.143519 11606 leader_election.cc:302] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3
I20260809 15:53:12.145308 11606 leader_election.cc:302] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:12.145608 14992 raft_consensus.cc:2781] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:12.146381 14992 raft_consensus.cc:482] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:12.147020 14966 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:12.147997 10978 leader_election.cc:334] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: RPC error from VoteRequest() call to peer c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685): Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.703s (SENT)
I20260809 15:53:12.147850 15007 consensus_queue.cc:1035] T 3d87bec6301b44c7b8dd46364e6110a8 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.016s
I20260809 15:53:12.146991 14992 raft_consensus.cc:3037] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:12.148415 14966 raft_consensus.cc:482] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:12.153788 14966 raft_consensus.cc:504] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.157522 14966 leader_election.cc:288] T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:12.121758 14918 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:12.148561 10978 leader_election.cc:302] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9
I20260809 15:53:12.161978 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "855d87c571d0427ba84900052dd9e2c0" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
W20260809 15:53:12.183303 14991 log.cc:924] Time spent T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.076s user 0.000s sys 0.001s
I20260809 15:53:12.183924 14845 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:12.184159 14805 leader_election.cc:288] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:12.217306 11637 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "c7aa762318cb4e97875333f10679a9f9" is_pre_election: true
I20260809 15:53:12.185200 14876 leader_election.cc:288] T 06c98b3690e54b19a8bc4b4366fce35a P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:12.119962 14407 leader_election.cc:288] T 87c018b6b9eb45cab9c8349a86ffb7d2 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:12.109229 15001 raft_consensus.cc:482] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:12.191560 10930 raft_consensus.cc:2454] T 855d87c571d0427ba84900052dd9e2c0 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 7.
W20260809 15:53:12.195528 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 47546.3us
I20260809 15:53:12.243867 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:12.244233 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:12.244851 10848 raft_consensus.cc:2454] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 6.
I20260809 15:53:12.251088 10930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:12.163873 14247 leader_election.cc:288] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:12.187851 14967 raft_consensus.cc:2781] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 9 FOLLOWER]: Leader election won for term 9
I20260809 15:53:12.194404 14650 raft_consensus.cc:2726] T 908409e5747b470cabbc8aed443397f7 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:12.215018 14871 raft_consensus.cc:2726] T 8777c94cbe134bf3aa222691d76ba470 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 LEADER]: Leader election lost for term 5. Reason: could not achieve majority
I20260809 15:53:12.255731 14785 raft_consensus.cc:686] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [term 7 LEADER]: Becoming Leader. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:12.189973 15005 raft_consensus.cc:2781] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Leader election won for term 9
W20260809 15:53:12.257637 14928 log.cc:924] Time spent T e42f46986036495ab1735d596ab06d6d P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.056s user 0.001s sys 0.000s
I20260809 15:53:12.211822 14918 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:53:12.265298 14980 log.cc:924] Time spent T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.359s user 0.002s sys 0.000s
I20260809 15:53:12.283789 10978 leader_election.cc:302] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d74944fa9fa14ec78b0640ea709ff3aa; no voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9
W20260809 15:53:12.271452 14492 consensus_meta.cc:287] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536: Time spent flushing consensus metadata: real 0.664s user 0.003s sys 0.011s
W20260809 15:53:12.275938 14652 raft_consensus.cc:922] Couldn't advance consensus term.: Illegal state: Can't advance term to: 8 current term: 8 is higher.
I20260809 15:53:12.193284 14990 consensus_queue.cc:1035] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.274s
I20260809 15:53:12.187696 14970 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
W20260809 15:53:12.295003 14987 log.cc:924] Time spent T f4d1cc07da5b49ae95998fabc2323d24 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.252s user 0.001s sys 0.000s
I20260809 15:53:12.282886 14760 leader_election.cc:288] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 11 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:12.295840 14892 log.cc:924] Time spent T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 2.523s user 0.000s sys 0.004s
I20260809 15:53:12.285463 15005 raft_consensus.cc:686] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 9 LEADER]: Becoming Leader. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:12.259807 14992 raft_consensus.cc:504] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:12.257984 15001 raft_consensus.cc:504] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.319335 14503 raft_consensus.cc:3032] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Stepping down as leader of term 7
I20260809 15:53:12.328558 14503 raft_consensus.cc:729] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 7 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:12.329245 14864 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:12.329885 14503 consensus_queue.cc:260] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 7.3, Last appended by leader: 3, Current term: 7, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:12.330039 14864 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.331840 14503 raft_consensus.cc:3037] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
W20260809 15:53:12.334818 14993 log.cc:924] Time spent T 87c018b6b9eb45cab9c8349a86ffb7d2 P 20730d75ad954a82b126148f8152a9f3: Append to log took a long time: real 0.203s user 0.000s sys 0.001s
W20260809 15:53:12.337436 14995 log.cc:924] Time spent T 9602fc7c9e814af09c87e4cb17fdb08d P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 0.170s user 0.000s sys 0.001s
I20260809 15:53:12.301777 14641 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:12.264797 14994 consensus_queue.cc:1035] T bf6504ce1c0d4241bfb97eb553c8df35 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.151s
I20260809 15:53:12.325332 14854 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:12.320690 14970 leader_election.cc:288] T 93303d409ab940409d87013868417907 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:53:12.331113 14955 log.cc:924] Time spent T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 1.000s user 0.002s sys 0.004s
I20260809 15:53:12.288044 14753 consensus_queue.cc:237] T e80d8e2921024e68a1b7c56fda99ee80 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 7.8, Last appended by leader: 8, Current term: 8, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.340776 14981 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:12.337529 14992 leader_election.cc:288] T 855d87c571d0427ba84900052dd9e2c0 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:12.349859 15005 consensus_queue.cc:237] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 8.6, Last appended by leader: 6, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:12.355731 15001 leader_election.cc:288] T c695a0d40c7e4b04a1f7a1113954a8e4 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:12.359683 14965 raft_consensus.cc:3032] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 15:53:12.370362 14965 raft_consensus.cc:729] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:12.374367 14965 consensus_queue.cc:260] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:12.377353 14965 raft_consensus.cc:3037] T 898f053882ab4598a37b00ed268ca6b3 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:53:12.428133 10844 raft_consensus.cc:2454] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 6.
I20260809 15:53:12.429606 10838 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54342 (request call id 907) took 1548 ms. Trace:
I20260809 15:53:12.430124 10838 rpcz_store.cc:276] 0809 15:53:10.881109 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:11.146918 (+265809us) service_pool.cc:227] Handling call
0809 15:53:11.242915 (+ 95997us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:11.247283 (+ 4368us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:12.429018 (+1181735us) spinlock_profiling.cc:243] Waited 95.6 ms on lock 0x7b540037d7d0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:12.429352 (+ 334us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":219984768}
I20260809 15:53:12.432447 10842 raft_consensus.cc:1261] T 908409e5747b470cabbc8aed443397f7 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 5 index: 7. Preceding OpId from leader: term: 6 index: 8. (index mismatch)
I20260809 15:53:12.432222 10838 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:12.433579 10838 raft_consensus.cc:3037] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Advancing to term 11
I20260809 15:53:12.433887 10842 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:45594 (request call id 2418) took 2339 ms. Trace:
W20260809 15:53:12.436249 10844 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:54342 (request call id 895) took 2391 ms (client timeout 1560 ms). Trace:
I20260809 15:53:12.434346 10842 rpcz_store.cc:276] 0809 15:53:10.094971 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:10.236627 (+141656us) service_pool.cc:227] Handling call
0809 15:53:12.431505 (+2194878us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:12.432929 (+ 1424us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:12.433664 (+ 735us) spinlock_profiling.cc:243] Waited 2.15 s on lock 0x7b54001716d0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:12.433808 (+ 144us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":4950568832}
W20260809 15:53:12.437012 10844 rpcz_store.cc:269] 0809 15:53:10.045020 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:10.046224 (+ 1204us) service_pool.cc:227] Handling call
0809 15:53:12.431195 (+2384971us) spinlock_profiling.cc:243] Waited 2.31 s on lock 0x7b54001716d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:12.436136 (+ 4941us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5312232192}
I20260809 15:53:12.438060 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cfa61e868cfb4915b0572f8e33745f43" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:12.438527 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:12.439020 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "279308d651cf45ff86edaae4483bcb79" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:12.439637 10842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:12.439925 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:12.440613 10842 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P a53441872c0c416b8e2f26f52399f536 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:53:12.441021 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:12.444941 10841 raft_consensus.cc:1226] T 9a5ecb5bb6e347a6921a7c894853e92e P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:12.362155 14861 raft_consensus.cc:504] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.367055 14918 leader_election.cc:288] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:12.484390 11018 raft_consensus.cc:2454] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 8.
I20260809 15:53:12.521111 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d87bec6301b44c7b8dd46364e6110a8" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 5 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:53:12.522241 11635 thread.cc:636] wal-append [worker] (thread pool) Time spent creating pthread: real 0.644s user 0.012s sys 0.007s
W20260809 15:53:12.522856 11635 thread.cc:602] wal-append [worker] (thread pool) Time spent starting thread: real 0.666s user 0.012s sys 0.007s
I20260809 15:53:12.525656 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1f9b7d9e57934596b377ee8dc79ea23e" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 11 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:12.527765 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:12.529472 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:12.530301 14675 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:12.530844 14992 raft_consensus.cc:2726] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Leader election lost for term 9. Reason: could not achieve majority
I20260809 15:53:12.530905 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:12.531338 14675 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:12.533421 11010 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 10.
I20260809 15:53:12.349989 14917 raft_consensus.cc:504] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.535570 15014 raft_consensus.cc:482] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:12.536657 15014 raft_consensus.cc:504] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.539921 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:12.540807 10841 raft_consensus.cc:1226] T e42f46986036495ab1735d596ab06d6d P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 6. Current term is 7. Ops: []
I20260809 15:53:12.540998 10923 raft_consensus.cc:2379] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa in current term 10: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:12.542145 15014 leader_election.cc:288] T 0b9fb9f88a6847af87fef710d1a2793b P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:12.542649 14861 leader_election.cc:288] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:12.542654 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:12.547103 11012 raft_consensus.cc:1226] T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 LEADER]: Rejecting Update request from peer c7aa762318cb4e97875333f10679a9f9 for earlier term 4. Current term is 5. Ops: []
I20260809 15:53:12.547195 10922 raft_consensus.cc:1226] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 6. Current term is 7. Ops: []
I20260809 15:53:12.269832 14785 consensus_queue.cc:237] T 2edb972a2a164f00a942aa6248187fe8 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 5.7, Last appended by leader: 7, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.562228 10839 raft_consensus.cc:1226] T 5e8d12b5ef9e415cbcb25e62ab228546 P a53441872c0c416b8e2f26f52399f536 [term 11 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 10. Current term is 11. Ops: [10.10-10.10]
I20260809 15:53:12.562912 10926 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 930) took 2158 ms. Trace:
I20260809 15:53:12.563422 10926 rpcz_store.cc:276] 0809 15:53:10.404443 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:10.404769 (+ 326us) service_pool.cc:227] Handling call
0809 15:53:10.405150 (+ 381us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:10.406857 (+ 1707us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:10.406902 (+ 45us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:10.413428 (+ 6526us) log.cc:842] Serialized 26 byte log entry
0809 15:53:10.761196 (+347768us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:53:10.761233 (+ 37us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:12.562178 (+1800945us) spinlock_profiling.cc:243] Waited 1.35 ms on lock 0x7b540024c7d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:12.562257 (+ 79us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:12.562357 (+ 100us) raft_consensus.cc:1671] finished
0809 15:53:12.562383 (+ 26us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:12.562724 (+ 341us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":445,"raft.run_cpu_time_us":123,"raft.run_wall_time_us":7686,"spinlock_wait_cycles":3101568,"thread_start_us":312669,"threads_started":1,"wal-append.queue_time_us":624328}
I20260809 15:53:12.565407 10931 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:52758 (request call id 943) took 1279 ms. Trace:
I20260809 15:53:12.566809 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6dbe3aec3ad84a48b982e17d89d8b6ab" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:12.575083 11011 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2369) took 2099 ms. Trace:
I20260809 15:53:12.575462 10809 leader_election.cc:302] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a53441872c0c416b8e2f26f52399f536, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:12.576319 11016 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50542 (request call id 2565) took 2821 ms. Trace:
I20260809 15:53:12.580394 10921 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54236 (request call id 2186) took 1011 ms. Trace:
I20260809 15:53:12.580842 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:12.577292 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8975b3e7ddc544bc888dd61fffc5313b" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 4 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:12.551453 14917 leader_election.cc:288] T 6508f17228904310be3c95818973f3a1 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:12.577015 11016 rpcz_store.cc:276] 0809 15:53:09.754591 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:09.755091 (+ 500us) service_pool.cc:227] Handling call
0809 15:53:09.755406 (+ 315us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:10.124737 (+369331us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:12.575774 (+2451037us) spinlock_profiling.cc:243] Waited 291 ms on lock 0x7b54001ddb54. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:12.576103 (+ 329us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":668699904}
I20260809 15:53:12.565917 10931 rpcz_store.cc:276] 0809 15:53:11.285804 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:11.285923 (+ 119us) service_pool.cc:227] Handling call
0809 15:53:11.286237 (+ 314us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:11.487024 (+200787us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:11.487071 (+ 47us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:11.493282 (+ 6211us) log.cc:842] Serialized 26 byte log entry
0809 15:53:11.652488 (+159206us) raft_consensus.cc:1631] Marking committed up to 8
0809 15:53:11.652542 (+ 54us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:12.564677 (+912135us) spinlock_profiling.cc:243] Waited 200 ms on lock 0x7b540030a7d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:12.564752 (+ 75us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:12.564831 (+ 79us) raft_consensus.cc:1671] finished
0809 15:53:12.564843 (+ 12us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:12.565173 (+ 330us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":40426,"raft.run_cpu_time_us":149,"raft.run_wall_time_us":4384,"spinlock_wait_cycles":460896128,"thread_start_us":158888,"threads_started":1,"wal-append.queue_time_us":172346}
I20260809 15:53:12.585220 11014 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2365) took 2553 ms. Trace:
I20260809 15:53:12.585332 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b472f5780ac4f56a2443e8ccc49125f" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:12.576527 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6ba8250f4ef43bfbf8f517e86a131a1" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:12.580873 10921 rpcz_store.cc:276] 0809 15:53:11.570052 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:11.573028 (+ 2976us) service_pool.cc:227] Handling call
0809 15:53:11.573433 (+ 405us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:11.624094 (+ 50661us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:12.576656 (+952562us) spinlock_profiling.cc:243] Waited 17.9 ms on lock 0x7b540015bdd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:12.580183 (+ 3527us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":41141632}
I20260809 15:53:12.593672 11632 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41478 (request call id 1114) took 1818 ms. Trace:
I20260809 15:53:12.594640 11633 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 1086) took 1314 ms. Trace:
W20260809 15:53:12.595386 10811 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.753s user 0.015s sys 0.012s
W20260809 15:53:12.595855 10811 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.789s user 0.015s sys 0.012s
W20260809 15:53:12.596536 11639 rpcz_store.cc:267] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:41482 (request call id 839) took 3267 ms (client timeout 1802 ms). Trace:
W20260809 15:53:12.596988 11639 rpcz_store.cc:269] 0809 15:53:09.328514 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:09.328624 (+ 110us) service_pool.cc:227] Handling call
0809 15:53:12.595952 (+3267328us) spinlock_profiling.cc:243] Waited 1.18 s on lock 0x7b54006f45d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:12.596208 (+ 256us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2709166336}
I20260809 15:53:12.598855 11636 raft_consensus.cc:1226] T cfa61e868cfb4915b0572f8e33745f43 P c7aa762318cb4e97875333f10679a9f9 [term 8 LEADER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 7. Current term is 8. Ops: []
I20260809 15:53:12.595091 11633 rpcz_store.cc:276] 0809 15:53:11.279659 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:11.280533 (+ 874us) service_pool.cc:227] Handling call
0809 15:53:11.280775 (+ 242us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:11.288912 (+ 8137us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:12.594152 (+1305240us) spinlock_profiling.cc:243] Waited 5.7 ms on lock 0x7b5400533754. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:12.594451 (+ 299us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":13102464}
I20260809 15:53:12.594189 11632 rpcz_store.cc:276] 0809 15:53:10.774702 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:10.828150 (+ 53448us) service_pool.cc:227] Handling call
0809 15:53:10.840074 (+ 11924us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:11.640281 (+800207us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:12.592979 (+952698us) spinlock_profiling.cc:243] Waited 794 ms on lock 0x7b54005aebd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:12.593320 (+ 341us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":1825594112}
W20260809 15:53:12.602321 15016 log.cc:924] Time spent T 2b4306bfa5464b0dbf8301756455ba50 P d74944fa9fa14ec78b0640ea709ff3aa: Append to log took a long time: real 0.051s user 0.000s sys 0.002s
W20260809 15:53:12.599960 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 797521us
I20260809 15:53:12.608161 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87c018b6b9eb45cab9c8349a86ffb7d2" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:12.611712 11017 raft_consensus.cc:3037] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:12.617233 14675 leader_election.cc:288] T 9d3f35b0ae8e4b02b72f832ca1518464 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:12.576824 11011 rpcz_store.cc:276] 0809 15:53:10.476022 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:10.476169 (+ 147us) service_pool.cc:227] Handling call
0809 15:53:10.482474 (+ 6305us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:11.827370 (+1344896us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:12.574551 (+747181us) spinlock_profiling.cc:243] Waited 1.34 s on lock 0x7b540040b854. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:12.574855 (+ 304us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3091443072}
I20260809 15:53:12.620410 11011 raft_consensus.cc:1261] T 898f053882ab4598a37b00ed268ca6b3 P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Refusing update from remote peer 20730d75ad954a82b126148f8152a9f3: Log matching property violated. Preceding OpId in replica: term: 8 index: 6. Preceding OpId from leader: term: 9 index: 7. (index mismatch)
I20260809 15:53:12.622622 14730 raft_consensus.cc:482] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:12.623451 14730 raft_consensus.cc:504] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:12.585745 11014 rpcz_store.cc:276] 0809 15:53:10.031405 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:10.031965 (+ 560us) service_pool.cc:227] Handling call
0809 15:53:11.478599 (+1446634us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:11.478774 (+ 175us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:11.478798 (+ 24us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:11.478837 (+ 39us) raft_consensus.cc:1631] Marking committed up to 4
0809 15:53:11.478866 (+ 29us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:11.478915 (+ 49us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:12.584887 (+1105972us) spinlock_profiling.cc:243] Waited 1.45 s on lock 0x7b54003f8c50. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:12.585060 (+ 173us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":3326190464}
W20260809 15:53:12.630239 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 55047.8us
I20260809 15:53:12.648084 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8331c9254a504745ac450b9d41ea6ab7" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 6 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:12.656874 10926 raft_consensus.cc:2454] T 8975b3e7ddc544bc888dd61fffc5313b P 20730d75ad954a82b126148f8152a9f3 [term 4 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d74944fa9fa14ec78b0640ea709ff3aa in term 4.
I20260809 15:53:12.656781 10931 tablet_service.cc:1967] Received LeaderStepDown RPC: tablet_id: "fe21e0688ade405bb4590b9c256bd696"
dest_uuid: "20730d75ad954a82b126148f8152a9f3"
mode: GRACEFUL
new_leader_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
from {username='slave'} at 127.0.0.1:34958
I20260809 15:53:12.657486 14730 leader_election.cc:288] T f4d1cc07da5b49ae95998fabc2323d24 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:12.657665 10931 raft_consensus.cc:595] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Received request to transfer leadership to TS d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:12.658183 10931 raft_consensus.cc:603] T fe21e0688ade405bb4590b9c256bd696 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting request to transer leadership while not leader
I20260809 15:53:12.658200 14673 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:12.659202 14673 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:12.659896 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 227158us
I20260809 15:53:12.660848 10811 leader_election.cc:302] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:12.666395 14402 raft_consensus.cc:2726] T 3d87bec6301b44c7b8dd46364e6110a8 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Leader pre-election lost for term 7. Reason: could not achieve majority
I20260809 15:53:12.668521 14673 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:12.669703 11603 thread.cc:636] raft [worker] (thread pool) Time spent creating pthread: real 0.670s user 0.005s sys 0.013s
I20260809 15:53:12.674440 14877 raft_consensus.cc:3037] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:12.679157 14917 raft_consensus.cc:482] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:12.679740 10921 raft_consensus.cc:1261] T 9a5ecb5bb6e347a6921a7c894853e92e P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Refusing update from remote peer d74944fa9fa14ec78b0640ea709ff3aa: Log matching property violated. Preceding OpId in replica: term: 7 index: 3. Preceding OpId from leader: term: 8 index: 4. (index mismatch)
I20260809 15:53:12.679909 14917 raft_consensus.cc:504] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:12.681010 14946 log.cc:924] Time spent T 02f25d93293a40729fd751a19ab9adeb P a53441872c0c416b8e2f26f52399f536: Append to log took a long time: real 1.497s user 0.003s sys 0.000s
W20260809 15:53:12.686981 11603 thread.cc:602] raft [worker] (thread pool) Time spent starting thread: real 0.804s user 0.006s sys 0.013s
I20260809 15:53:12.713011 11010 raft_consensus.cc:2454] T 8331c9254a504745ac450b9d41ea6ab7 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 7.
W20260809 15:53:12.711004 14789 thread.cc:636] prepare [worker] (thread pool) Time spent creating pthread: real 0.593s user 0.002s sys 0.012s
I20260809 15:53:12.726116 11017 raft_consensus.cc:2454] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 7.
W20260809 15:53:12.729552 14642 consensus_meta.cc:287] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.792s user 0.000s sys 0.017s
I20260809 15:53:12.730206 14642 raft_consensus.cc:2726] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 11 FOLLOWER]: Leader pre-election lost for term 11. Reason: could not achieve majority
I20260809 15:53:12.712023 14730 raft_consensus.cc:482] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
W20260809 15:53:12.731657 14948 consensus_meta.cc:287] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 0.608s user 0.015s sys 0.005s
I20260809 15:53:12.731320 14730 raft_consensus.cc:504] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
W20260809 15:53:12.727996 14789 thread.cc:602] prepare [worker] (thread pool) Time spent starting thread: real 0.760s user 0.003s sys 0.012s
I20260809 15:53:12.663617 15012 consensus_queue.cc:1035] T f4d1cc07da5b49ae95998fabc2323d24 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Connected to new peer: Peer: permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 4, Time since last communication: 0.063s
W20260809 15:53:12.744908 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 84281.2us
I20260809 15:53:12.751354 14730 leader_election.cc:288] T b6d399a7394c4a59b90a308aedfdf899 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:12.754967 10838 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 11.
I20260809 15:53:12.758144 14966 raft_consensus.cc:2781] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Leader pre-election won for term 11
I20260809 15:53:12.758656 14966 raft_consensus.cc:482] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:12.759080 14966 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 10 FOLLOWER]: Advancing to term 11
I20260809 15:53:12.760670 14760 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:12.761487 14760 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.764299 14760 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
W20260809 15:53:12.765446 14638 consensus_meta.cc:287] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9: Time spent flushing consensus metadata: real 0.679s user 0.007s sys 0.011s
I20260809 15:53:12.772521 14947 consensus_queue.cc:1035] T 898f053882ab4598a37b00ed268ca6b3 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.001s
I20260809 15:53:12.781169 11634 raft_consensus.cc:2454] T 1f9b7d9e57934596b377ee8dc79ea23e P c7aa762318cb4e97875333f10679a9f9 [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 11.
I20260809 15:53:12.790690 14917 leader_election.cc:288] T 5b472f5780ac4f56a2443e8ccc49125f P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:12.732091 14948 raft_consensus.cc:504] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:12.797281 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "34001ced69f34c6f930f5218eef46524" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:12.813057 14983 raft_consensus.cc:482] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting leader election (detected failure of leader d74944fa9fa14ec78b0640ea709ff3aa)
I20260809 15:53:12.813741 14983 raft_consensus.cc:3037] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:53:12.781052 14803 consensus_queue.cc:1046] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 6.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.037s
I20260809 15:53:12.815174 14803 raft_consensus.cc:3032] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 15:53:12.815569 14803 raft_consensus.cc:729] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:12.822424 10891 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:12.823673 14949 raft_consensus.cc:2781] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Leader pre-election won for term 8
I20260809 15:53:12.824070 14949 raft_consensus.cc:482] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:12.824432 14949 raft_consensus.cc:3037] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:12.825153 9552 auto_leader_rebalancer.cc:388] table: test-workload, leader rebalance finish, leader transfer count: 6
I20260809 15:53:12.832979 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d3f35b0ae8e4b02b72f832ca1518464" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 9 candidate_status { last_received { term: 5 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
W20260809 15:53:12.835106 10809 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 66157.5us
I20260809 15:53:12.765918 14638 raft_consensus.cc:504] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.816671 14803 consensus_queue.cc:260] T 81af4312dd044dfa85b9067b2fe4974c P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 8.8, Last appended by leader: 8, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.841451 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b6d399a7394c4a59b90a308aedfdf899" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:12.842649 14847 consensus_queue.cc:1046] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 4, Time since last communication: 0.001s
I20260809 15:53:12.843950 10811 leader_election.cc:302] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 8 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:12.844362 14638 leader_election.cc:288] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:12.843936 15023 raft_consensus.cc:3032] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 15:53:12.845667 15023 raft_consensus.cc:729] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: a53441872c0c416b8e2f26f52399f536, State: Running, Role: LEADER
I20260809 15:53:12.850723 11017 raft_consensus.cc:1226] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 6. Current term is 7. Ops: []
W20260809 15:53:12.849149 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 127420us
I20260809 15:53:12.842667 11015 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:39978 (request call id 864) took 1329 ms. Trace:
I20260809 15:53:12.856958 11015 rpcz_store.cc:276] 0809 15:53:11.512936 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:11.516330 (+ 3394us) service_pool.cc:227] Handling call
0809 15:53:11.516680 (+ 350us) raft_consensus.cc:1457] Updating replica for 1 ops
0809 15:53:11.516975 (+ 295us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:11.517023 (+ 48us) raft_consensus.cc:1525] Triggering prepare for 1 ops
0809 15:53:11.517600 (+ 577us) log.cc:842] Serialized 26 byte log entry
0809 15:53:12.841577 (+1323977us) spinlock_profiling.cc:243] Waited 995 us on lock 0x7b1800493300. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f796f46a8b1 00007f796f477661 00007f796918282c 00007f79665ad9dc 00007f796f3e4a39 00007f79665ca8bb 00007f79665c8cbf 00007f79665cae22 00007f79691745bb 00007f796917a063 00007f796917a411
0809 15:53:12.841941 (+ 364us) raft_consensus.cc:1631] Marking committed up to 5
0809 15:53:12.841997 (+ 56us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:12.842107 (+ 110us) raft_consensus.cc:1654] Waiting on the replicates to finish logging
0809 15:53:12.842188 (+ 81us) raft_consensus.cc:1671] finished
0809 15:53:12.842200 (+ 12us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:12.842465 (+ 265us) inbound_call.cc:173] Queueing success response
Metrics: {"raft.queue_time_us":52396,"raft.run_cpu_time_us":103,"raft.run_wall_time_us":2782,"spinlock_wait_cycles":2288512,"thread_start_us":58846,"threads_started":1,"wal-append.queue_time_us":153560}
I20260809 15:53:12.845961 10811 leader_election.cc:302] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a53441872c0c416b8e2f26f52399f536; no voters: 20730d75ad954a82b126148f8152a9f3, d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:12.849191 11011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:12.826103 14917 raft_consensus.cc:482] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
W20260809 15:53:12.849220 14801 consensus_meta.cc:287] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3: Time spent flushing consensus metadata: real 1.297s user 0.006s sys 0.016s
I20260809 15:53:12.849233 14612 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:12.846297 14760 raft_consensus.cc:2726] T 6dbe3aec3ad84a48b982e17d89d8b6ab P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Leader pre-election lost for term 8. Reason: could not achieve majority
I20260809 15:53:12.876964 14612 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 13 FOLLOWER]: Advancing to term 14
I20260809 15:53:12.868448 14917 raft_consensus.cc:504] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:12.884054 10927 raft_consensus.cc:2396] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for term 7 because replica has last-logged OpId of term: 5 index: 6, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 5.
I20260809 15:53:12.890982 11011 raft_consensus.cc:2379] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 8: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
W20260809 15:53:12.883567 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 569235us
I20260809 15:53:12.872689 14847 raft_consensus.cc:482] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:12.887770 11636 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:41488 (request call id 1074) took 2787 ms. Trace:
I20260809 15:53:12.893962 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:12.896662 10978 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 7 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:12.899044 14701 raft_consensus.cc:3037] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:12.885577 14948 leader_election.cc:288] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 election: Requested vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:12.891798 15026 consensus_queue.cc:1046] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 6.8, Next index: 9, Last known committed idx: 8, Time since last communication: 0.022s
I20260809 15:53:12.901238 14348 raft_consensus.cc:482] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:12.901890 15026 raft_consensus.cc:3032] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 6 LEADER]: Stepping down as leader of term 6
I20260809 15:53:12.902315 14487 raft_consensus.cc:482] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:12.903021 10927 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.RequestConsensusVote from 127.0.0.1:52758 (request call id 948) took 1166 ms. Trace:
I20260809 15:53:12.903458 10927 rpcz_store.cc:276] 0809 15:53:11.736385 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:11.736829 (+ 444us) service_pool.cc:227] Handling call
0809 15:53:12.902789 (+1165960us) spinlock_profiling.cc:243] Waited 1.13 s on lock 0x7b540021b1d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f72ae 00007f796e434c27 00007f7967b1cc04 00007f7967b1cb92 00007f7967b1cac1 00007f7967b1ca3c 00007f7967b1bcf2 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:12.902925 (+ 136us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":2607333376}
I20260809 15:53:12.904485 14917 leader_election.cc:288] T 9602fc7c9e814af09c87e4cb17fdb08d P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:12.905701 14847 raft_consensus.cc:504] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.908054 14847 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:12.894608 11636 rpcz_store.cc:276] 0809 15:53:10.099870 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:10.237374 (+137504us) service_pool.cc:227] Handling call
0809 15:53:12.598248 (+2360874us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:12.599420 (+ 1172us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:12.887514 (+288094us) spinlock_profiling.cc:243] Waited 2.32 s on lock 0x7b54006f45d0. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f2a09 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86 00007f796791c72f
0809 15:53:12.887667 (+ 153us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":5329680640}
I20260809 15:53:12.908651 14864 leader_election.cc:288] T f866570ff0c5456ba186b9234cb692c8 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:12.917568 14967 raft_consensus.cc:686] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [term 9 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:53:12.918252 11009 raft_consensus.cc:1261] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Refusing update from remote peer a53441872c0c416b8e2f26f52399f536: Log matching property violated. Preceding OpId in replica: term: 7 index: 5. Preceding OpId from leader: term: 8 index: 5. (term mismatch)
I20260809 15:53:12.918701 11009 pending_rounds.cc:76] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa: Aborting all ops after (but not including) 4
I20260809 15:53:12.919088 11009 pending_rounds.cc:97] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa: Aborting uncommitted NO_OP operation due to leader change: 7.5
I20260809 15:53:12.919457 11009 raft_consensus.cc:2866] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
W20260809 15:53:12.920099 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 433491us
I20260809 15:53:12.920398 14872 consensus_queue.cc:1046] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260809 15:53:12.921168 11014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f4d1cc07da5b49ae95998fabc2323d24" candidate_uuid: "20730d75ad954a82b126148f8152a9f3" candidate_term: 11 candidate_status { last_received { term: 8 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:12.921223 11606 leader_election.cc:302] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 11 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9; no voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536
I20260809 15:53:12.921968 14872 raft_consensus.cc:3032] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Stepping down as leader of term 4
I20260809 15:53:12.922384 14872 raft_consensus.cc:729] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 4 LEADER]: Becoming Follower/Learner. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
W20260809 15:53:12.923535 11604 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.9.84.3:39223, user_credentials={real_user=slave}} blocked reactor thread for 341877us
I20260809 15:53:12.925148 14886 raft_consensus.cc:482] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:12.925756 14886 raft_consensus.cc:504] T 9d3f35b0ae8e4b02b72f832ca1518464 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:12.926394 14254 raft_consensus.cc:482] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:12.926486 14948 raft_consensus.cc:482] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:12.927117 14948 raft_consensus.cc:504] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.927117 14254 raft_consensus.cc:504] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.930223 14254 leader_election.cc:288] T 8777c94cbe134bf3aa222691d76ba470 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:12.930616 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:12.930738 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:12.903033 14487 raft_consensus.cc:504] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.932683 11017 raft_consensus.cc:2354] T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for earlier term 6. Current term is 7.
I20260809 15:53:12.933449 15023 consensus_queue.cc:260] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 8.6, Last appended by leader: 6, Current term: 8, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:12.901940 14348 raft_consensus.cc:504] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.934888 14877 raft_consensus.cc:482] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:12.935189 15023 raft_consensus.cc:3037] T 87c018b6b9eb45cab9c8349a86ffb7d2 P a53441872c0c416b8e2f26f52399f536 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:53:12.935587 14877 raft_consensus.cc:504] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Starting pre-election with config: opid_index: 5 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.936463 14728 raft_consensus.cc:482] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260809 15:53:12.930559 10811 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.2:32783, user_credentials={real_user=slave}} blocked reactor thread for 84778.8us
I20260809 15:53:12.937022 14728 raft_consensus.cc:504] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.937232 14348 leader_election.cc:288] T c695a0d40c7e4b04a1f7a1113954a8e4 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:12.939833 11013 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2382) took 1321 ms. Trace:
I20260809 15:53:12.940287 14948 leader_election.cc:288] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 7 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:12.919139 14967 consensus_queue.cc:237] T 11cab658217a44bebba11963c723e3dc P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 7.7, Last appended by leader: 7, Current term: 9, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.940236 11013 rpcz_store.cc:276] 0809 15:53:11.618132 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:11.620284 (+ 2152us) service_pool.cc:227] Handling call
0809 15:53:11.620559 (+ 275us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:11.652222 (+ 31663us) raft_consensus.cc:1520] Early marking committed up to index 0
0809 15:53:11.652250 (+ 28us) raft_consensus.cc:1525] Triggering prepare for 0 ops
0809 15:53:11.652322 (+ 72us) raft_consensus.cc:1631] Marking committed up to 6
0809 15:53:11.652365 (+ 43us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:12.939441 (+1287076us) spinlock_profiling.cc:243] Waited 31.5 ms on lock 0x7b540004ddd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:12.939507 (+ 66us) raft_consensus.cc:1677] UpdateReplicas() finished
0809 15:53:12.939665 (+ 158us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":72360192}
I20260809 15:53:12.943909 10924 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:54224 (request call id 2184) took 1295 ms. Trace:
I20260809 15:53:12.944424 10924 rpcz_store.cc:276] 0809 15:53:11.648633 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:11.648766 (+ 133us) service_pool.cc:227] Handling call
0809 15:53:11.652978 (+ 4212us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:11.746294 (+ 93316us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:12.943313 (+1197019us) spinlock_profiling.cc:243] Waited 78.4 ms on lock 0x7b54000a8cd4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:12.943656 (+ 343us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":180241664}
I20260809 15:53:12.945616 14487 leader_election.cc:288] T 81af4312dd044dfa85b9067b2fe4974c P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:12.946606 14965 raft_consensus.cc:2726] T 8331c9254a504745ac450b9d41ea6ab7 P a53441872c0c416b8e2f26f52399f536 [term 7 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority
I20260809 15:53:12.923362 14872 consensus_queue.cc:260] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 4.7, Last appended by leader: 7, Current term: 4, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.902434 15026 raft_consensus.cc:729] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [term 6 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:12.937662 14877 leader_election.cc:288] T bf6504ce1c0d4241bfb97eb553c8df35 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 11 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
W20260809 15:53:12.918601 15022 log.cc:924] Time spent T 6d4d6dee78914014adab40a391b02955 P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.250s user 0.000s sys 0.001s
I20260809 15:53:12.932698 10848 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 10 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:12.935684 14997 raft_consensus.cc:482] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:12.949939 14374 raft_consensus.cc:3037] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Advancing to term 8
W20260809 15:53:12.952562 10978 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 58552.5us
I20260809 15:53:12.962926 11604 leader_election.cc:302] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 4 voters: 3 yes votes; 0 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:12.963551 14728 leader_election.cc:288] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:12.964505 14728 raft_consensus.cc:2781] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election won for term 9
I20260809 15:53:12.965112 11604 leader_election.cc:302] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:12.967185 14753 consensus_queue.cc:1035] T 369529a0023c4cf4a709bb1a480052b4 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.001s
I20260809 15:53:12.954816 14997 raft_consensus.cc:504] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.970853 14997 leader_election.cc:288] T dd8bcd39f37d403f9d8d8f43d870a878 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:12.971740 15008 raft_consensus.cc:2781] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election won for term 7
I20260809 15:53:12.972435 15008 raft_consensus.cc:686] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [term 7 LEADER]: Becoming Leader. State: Replica: c7aa762318cb4e97875333f10679a9f9, State: Running, Role: LEADER
I20260809 15:53:12.975857 14959 consensus_queue.cc:1046] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 4.7, Next index: 8, Last known committed idx: 7, Time since last communication: 0.018s
I20260809 15:53:12.976639 14959 consensus_queue.cc:1225] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [NON_LEADER]: Peer d74944fa9fa14ec78b0640ea709ff3aa log is divergent from this leader: its last log entry 5.8 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 7
I20260809 15:53:12.965190 14728 raft_consensus.cc:482] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:12.964540 15026 consensus_queue.cc:260] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P 20730d75ad954a82b126148f8152a9f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 6, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:12.990815 10927 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "06c98b3690e54b19a8bc4b4366fce35a" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 8 candidate_status { last_received { term: 7 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:12.992036 10927 raft_consensus.cc:2379] T 06c98b3690e54b19a8bc4b4366fce35a P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate a53441872c0c416b8e2f26f52399f536 in current term 8: Already voted for candidate 20730d75ad954a82b126148f8152a9f3 in this term.
I20260809 15:53:12.997566 14851 leader_election.cc:288] T 11cab658217a44bebba11963c723e3dc P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:12.998638 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8feeb57bf5cf4fd1bb17ab7f575222d0" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 9 candidate_status { last_received { term: 8 index: 9 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:12.973779 15008 consensus_queue.cc:237] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P c7aa762318cb4e97875333f10679a9f9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 6.8, Last appended by leader: 8, Current term: 7, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:13.017179 14789 raft_consensus.cc:482] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:13.017691 10923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:12.999756 10924 raft_consensus.cc:1226] T 93303d409ab940409d87013868417907 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 6. Current term is 7. Ops: []
I20260809 15:53:13.018407 10923 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P 20730d75ad954a82b126148f8152a9f3 [term 10 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 10.
I20260809 15:53:13.018877 10921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a89bcd329414d49a3bfdf5ac68523e9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 6 candidate_status { last_received { term: 5 index: 6 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:13.019624 10921 raft_consensus.cc:2454] T 6a89bcd329414d49a3bfdf5ac68523e9 P 20730d75ad954a82b126148f8152a9f3 [term 6 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 6.
I20260809 15:53:13.019989 10923 raft_consensus.cc:1226] T dd8bcd39f37d403f9d8d8f43d870a878 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Rejecting Update request from peer a53441872c0c416b8e2f26f52399f536 for earlier term 6. Current term is 7. Ops: []
I20260809 15:53:13.000494 10931 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e42f46986036495ab1735d596ab06d6d" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:13.022027 14966 raft_consensus.cc:504] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 11 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:13.023073 14935 raft_consensus.cc:482] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:12.986958 14728 raft_consensus.cc:3037] T 06c98b3690e54b19a8bc4b4366fce35a P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Advancing to term 9
I20260809 15:53:13.003798 10920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02f25d93293a40729fd751a19ab9adeb" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 7 candidate_status { last_received { term: 6 index: 7 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3" is_pre_election: true
I20260809 15:53:13.005777 11013 raft_consensus.cc:1226] T 6dbe3aec3ad84a48b982e17d89d8b6ab P d74944fa9fa14ec78b0640ea709ff3aa [term 9 LEADER]: Rejecting Update request from peer 20730d75ad954a82b126148f8152a9f3 for earlier term 8. Current term is 9. Ops: []
I20260809 15:53:13.005905 14674 raft_consensus.cc:482] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader c7aa762318cb4e97875333f10679a9f9)
I20260809 15:53:13.007207 10893 leader_election.cc:302] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 14 pre-election: Election decided. Result: candidate won. Election summary: received 4 responses out of 4 voters: 3 yes votes; 1 no votes. yes voters: 20730d75ad954a82b126148f8152a9f3, a53441872c0c416b8e2f26f52399f536, c7aa762318cb4e97875333f10679a9f9; no voters: d74944fa9fa14ec78b0640ea709ff3aa
I20260809 15:53:13.007671 11009 rpcz_store.cc:275] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.0.1:50528 (request call id 2349) took 3879 ms. Trace:
I20260809 15:53:13.027367 11009 rpcz_store.cc:276] 0809 15:53:09.128083 (+ 0us) service_pool.cc:170] Inserting onto call queue
0809 15:53:09.128208 (+ 125us) service_pool.cc:227] Handling call
0809 15:53:09.128509 (+ 301us) raft_consensus.cc:1457] Updating replica for 0 ops
0809 15:53:12.920097 (+3791588us) raft_consensus.cc:1683] Filling consensus response to leader.
0809 15:53:13.007359 (+ 87262us) spinlock_profiling.cc:243] Waited 3.79 s on lock 0x7b54000163d4. stack: 00007f7966599033 00007f7966598f30 00007f7965ef7534 00007f796fc038b4 00007f796fc03849 00007f7969478471 00007f79694f3f51 00007f79694f299e 00007f796e4342e9 00007f7967b1a684 00007f7967b1a612 00007f7967b1a541 00007f7967b1a4bc 00007f7967b19772 00007f796791d94c 00007f796791cd86
0809 15:53:13.007596 (+ 237us) inbound_call.cc:173] Queueing success response
Metrics: {"spinlock_wait_cycles":8714087808}
I20260809 15:53:13.007462 14951 consensus_queue.cc:1046] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: INVALID_TERM, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 15:53:12.983719 14917 raft_consensus.cc:2741] T 5dc95cd5fbb945e7af0f89cf2ec436c9 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader pre-election decision vote started in defunct term 6: won
W20260809 15:53:13.011089 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 90037.2us
I20260809 15:53:13.016285 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5e8d12b5ef9e415cbcb25e62ab228546" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 11 candidate_status { last_received { term: 9 index: 9 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"
I20260809 15:53:13.033705 10848 raft_consensus.cc:2354] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate d74944fa9fa14ec78b0640ea709ff3aa for earlier term 10. Current term is 11.
I20260809 15:53:13.023705 14935 raft_consensus.cc:504] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:13.013628 14612 raft_consensus.cc:504] T 0b2b0df042e44725a7c91deae3df1956 P d74944fa9fa14ec78b0640ea709ff3aa [term 14 FOLLOWER]: Starting leader election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } attrs { replace: true } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:13.032708 14997 raft_consensus.cc:3037] T 5e8d12b5ef9e415cbcb25e62ab228546 P c7aa762318cb4e97875333f10679a9f9 [term 10 FOLLOWER]: Advancing to term 11
I20260809 15:53:13.026127 14966 leader_election.cc:288] T 4ff56f55843f41598bce93ad8056bcb9 P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 11 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:13.017867 14789 raft_consensus.cc:504] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:13.092850 11013 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ff56f55843f41598bce93ad8056bcb9" candidate_uuid: "a53441872c0c416b8e2f26f52399f536" candidate_term: 11 candidate_status { last_received { term: 9 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa"
I20260809 15:53:13.096724 11013 raft_consensus.cc:3037] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 10 FOLLOWER]: Advancing to term 11
I20260809 15:53:13.095016 11604 leader_election.cc:398] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Vote denied by peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223) with higher term. Message: Invalid argument: T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for earlier term 6. Current term is 7.
I20260809 15:53:13.097410 11604 leader_election.cc:401] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 election: Cancelling election due to peer responding with higher term
I20260809 15:53:13.107628 14832 consensus_queue.cc:1046] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 }, Status: INVALID_TERM, Last received: 8.6, Next index: 7, Last known committed idx: 5, Time since last communication: 0.037s
I20260809 15:53:13.107931 14983 raft_consensus.cc:504] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:13.108585 14832 consensus_queue.cc:1225] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Peer d74944fa9fa14ec78b0640ea709ff3aa log is divergent from this leader: its last log entry 9.7 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5
W20260809 15:53:13.095264 10811 proxy.cc:239] Call had error, refreshing address and retrying: Timed out: RequestConsensusVote RPC to 127.9.84.4:37685 timed out after 1.802s (SENT) [suppressed 10 similar messages]
I20260809 15:53:13.113103 14654 raft_consensus.cc:2781] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 13 FOLLOWER]: Leader pre-election won for term 14
I20260809 15:53:13.113296 14961 raft_consensus.cc:3037] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 6 FOLLOWER]: Advancing to term 7
I20260809 15:53:13.113682 14654 raft_consensus.cc:482] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 13 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 15:53:13.114200 14654 raft_consensus.cc:3037] T 0b2b0df042e44725a7c91deae3df1956 P 20730d75ad954a82b126148f8152a9f3 [term 13 FOLLOWER]: Advancing to term 14
I20260809 15:53:13.114984 14967 raft_consensus.cc:482] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:13.115675 14967 raft_consensus.cc:504] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:13.117547 14872 raft_consensus.cc:3037] T 2b4306bfa5464b0dbf8301756455ba50 P c7aa762318cb4e97875333f10679a9f9 [term 4 FOLLOWER]: Advancing to term 5
I20260809 15:53:13.118346 14455 raft_consensus.cc:3037] T 70c84efc43004b7b8ef2f9b869250509 P 20730d75ad954a82b126148f8152a9f3 [term 7 FOLLOWER]: Advancing to term 8
I20260809 15:53:13.118290 14967 leader_election.cc:288] T 70c84efc43004b7b8ef2f9b869250509 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:13.119078 14967 raft_consensus.cc:482] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election (detected failure of leader 20730d75ad954a82b126148f8152a9f3)
I20260809 15:53:13.073048 14674 raft_consensus.cc:504] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:13.119921 14967 raft_consensus.cc:504] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [term 5 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:13.152100 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:13.109612 14949 raft_consensus.cc:504] T 8331c9254a504745ac450b9d41ea6ab7 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:13.160970 11010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c695a0d40c7e4b04a1f7a1113954a8e4" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 2 index: 5 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:13.164170 10839 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93303d409ab940409d87013868417907" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 7 candidate_status { last_received { term: 6 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
W20260809 15:53:13.175669 10893 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.4:37685, user_credentials={real_user=slave}} blocked reactor thread for 168599us
I20260809 15:53:13.176817 10841 raft_consensus.cc:1226] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [term 9 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 8. Current term is 9. Ops: []
I20260809 15:53:13.178247 14951 raft_consensus.cc:482] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
W20260809 15:53:13.178709 14892 log.cc:924] Time spent T e5757bc9ac9441c3bdfa6c5b2b743b6d P c7aa762318cb4e97875333f10679a9f9: Append to log took a long time: real 0.066s user 0.001s sys 0.000s
I20260809 15:53:13.178929 14951 raft_consensus.cc:504] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } }
I20260809 15:53:13.181046 14951 leader_election.cc:288] T 279308d651cf45ff86edaae4483bcb79 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783)
I20260809 15:53:13.182956 14674 leader_election.cc:288] T 34001ced69f34c6f930f5218eef46524 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 8 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:13.178287 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6508f17228904310be3c95818973f3a1" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:13.187233 11009 raft_consensus.cc:3032] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 LEADER]: Stepping down as leader of term 5
I20260809 15:53:13.187546 14888 raft_consensus.cc:482] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 15:53:13.187705 11009 raft_consensus.cc:729] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 LEADER]: Becoming Follower/Learner. State: Replica: d74944fa9fa14ec78b0640ea709ff3aa, State: Running, Role: LEADER
I20260809 15:53:13.188174 14888 raft_consensus.cc:504] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:13.188869 11009 consensus_queue.cc:260] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 5.6, Last appended by leader: 6, Current term: 5, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" member_type: VOTER last_known_addr { host: "127.9.84.3" port: 39223 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:13.170977 11016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "81af4312dd044dfa85b9067b2fe4974c" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 8 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:13.175561 11015 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13ef36e0f4c040a7963771a1dcb2a286" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 9 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:13.197158 11016 raft_consensus.cc:2379] T 81af4312dd044dfa85b9067b2fe4974c P d74944fa9fa14ec78b0640ea709ff3aa [term 9 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 9: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:13.199546 14851 raft_consensus.cc:482] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election (detected failure of leader a53441872c0c416b8e2f26f52399f536)
I20260809 15:53:13.200204 14851 raft_consensus.cc:504] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [term 8 FOLLOWER]: Starting pre-election with config: opid_index: 4 OBSOLETE_local: false peers { permanent_uuid: "a53441872c0c416b8e2f26f52399f536" member_type: VOTER last_known_addr { host: "127.9.84.1" port: 46615 } } peers { permanent_uuid: "20730d75ad954a82b126148f8152a9f3" member_type: VOTER last_known_addr { host: "127.9.84.2" port: 32783 } } peers { permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false } }
I20260809 15:53:13.201464 14666 consensus_queue.cc:1035] T 5e8d12b5ef9e415cbcb25e62ab228546 P 20730d75ad954a82b126148f8152a9f3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7aa762318cb4e97875333f10679a9f9" member_type: VOTER last_known_addr { host: "127.9.84.4" port: 37685 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 10, Last known committed idx: 8, Time since last communication: 0.001s
I20260809 15:53:13.203107 14851 leader_election.cc:288] T b6ba8250f4ef43bfbf8f517e86a131a1 P 20730d75ad954a82b126148f8152a9f3 [CANDIDATE]: Term 9 pre-election: Requested pre-vote from peers a53441872c0c416b8e2f26f52399f536 (127.9.84.1:46615), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:13.205510 11017 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "70c84efc43004b7b8ef2f9b869250509" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 8 candidate_status { last_received { term: 7 index: 8 } } ignore_live_leader: false dest_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" is_pre_election: true
I20260809 15:53:13.193560 10840 raft_consensus.cc:1226] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 5. Current term is 6. Ops: [5.6-5.6]
I20260809 15:53:13.210036 11015 raft_consensus.cc:2454] T 13ef36e0f4c040a7963771a1dcb2a286 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c7aa762318cb4e97875333f10679a9f9 in term 8.
I20260809 15:53:13.224344 10841 raft_consensus.cc:1226] T 1f9b7d9e57934596b377ee8dc79ea23e P a53441872c0c416b8e2f26f52399f536 [term 11 FOLLOWER]: Rejecting Update request from peer d74944fa9fa14ec78b0640ea709ff3aa for earlier term 10. Current term is 11. Ops: [10.8-10.8]
I20260809 15:53:13.225584 11641 raft_consensus.cc:2454] T e80d8e2921024e68a1b7c56fda99ee80 P c7aa762318cb4e97875333f10679a9f9 [term 8 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20730d75ad954a82b126148f8152a9f3 in term 8.
W20260809 15:53:13.226207 11606 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.9.84.1:46615, user_credentials={real_user=slave}} blocked reactor thread for 103998us
I20260809 15:53:13.226564 14967 leader_election.cc:288] T 7c8eaf50bcb3457bb4b1e30406cc349f P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:13.230306 14983 leader_election.cc:288] T 220f58b662044f089f2be802ed24284a P a53441872c0c416b8e2f26f52399f536 [CANDIDATE]: Term 9 election: Requested vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223)
I20260809 15:53:13.234841 14919 raft_consensus.cc:3032] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 8 LEADER]: Stepping down as leader of term 8
I20260809 15:53:13.235258 14919 raft_consensus.cc:729] T 6dbe3aec3ad84a48b982e17d89d8b6ab P 20730d75ad954a82b126148f8152a9f3 [term 8 LEADER]: Becoming Follower/Learner. State: Replica: 20730d75ad954a82b126148f8152a9f3, State: Running, Role: LEADER
I20260809 15:53:13.235654 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9602fc7c9e814af09c87e4cb17fdb08d" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 8 index: 4 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:13.237329 14789 leader_election.cc:288] T 974451500a6e434f8d407d9f178ab710 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 5 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:13.237521 14961 raft_consensus.cc:2726] T 2edb972a2a164f00a942aa6248187fe8 P c7aa762318cb4e97875333f10679a9f9 [term 7 FOLLOWER]: Leader election lost for term 6. Reason: Vote denied by peer d74944fa9fa14ec78b0640ea709ff3aa (127.9.84.3:39223) with higher term. Message: Invalid argument: T 2edb972a2a164f00a942aa6248187fe8 P d74944fa9fa14ec78b0640ea709ff3aa [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 for earlier term 6. Current term is 7.
I20260809 15:53:13.238449 14935 leader_election.cc:288] T 0f891fe62df749108fa553cb0661a874 P d74944fa9fa14ec78b0640ea709ff3aa [CANDIDATE]: Term 6 pre-election: Requested pre-vote from peers 20730d75ad954a82b126148f8152a9f3 (127.9.84.2:32783), c7aa762318cb4e97875333f10679a9f9 (127.9.84.4:37685)
I20260809 15:53:13.242252 11013 raft_consensus.cc:2454] T 4ff56f55843f41598bce93ad8056bcb9 P d74944fa9fa14ec78b0640ea709ff3aa [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a53441872c0c416b8e2f26f52399f536 in term 11.
I20260809 15:53:13.242851 11017 raft_consensus.cc:2379] T 70c84efc43004b7b8ef2f9b869250509 P d74944fa9fa14ec78b0640ea709ff3aa [term 8 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7aa762318cb4e97875333f10679a9f9 in current term 8: Already voted for candidate d74944fa9fa14ec78b0640ea709ff3aa in this term.
I20260809 15:53:13.243999 11009 raft_consensus.cc:3037] T 6d4d6dee78914014adab40a391b02955 P d74944fa9fa14ec78b0640ea709ff3aa [term 5 FOLLOWER]: Advancing to term 6
I20260809 15:53:13.249378 11604 leader_election.cc:302] T 13ef36e0f4c040a7963771a1dcb2a286 P c7aa762318cb4e97875333f10679a9f9 [CANDIDATE]: Term 9 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c7aa762318cb4e97875333f10679a9f9, d74944fa9fa14ec78b0640ea709ff3aa; no voters:
I20260809 15:53:13.251822 10844 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8777c94cbe134bf3aa222691d76ba470" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 4 index: 6 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:13.257385 10839 raft_consensus.cc:1261] T 6d4d6dee78914014adab40a391b02955 P a53441872c0c416b8e2f26f52399f536 [term 6 FOLLOWER]: Refusing update from remote peer c7aa762318cb4e97875333f10679a9f9: Log matching property violated. Preceding OpId in replica: term: 4 index: 5. Preceding OpId from leader: term: 6 index: 6. (index mismatch)
I20260809 15:53:13.258045 10840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2edb972a2a164f00a942aa6248187fe8" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 6 candidate_status { last_received { term: 5 index: 7 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536"
I20260809 15:53:13.262364 10841 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f866570ff0c5456ba186b9234cb692c8" candidate_uuid: "d74944fa9fa14ec78b0640ea709ff3aa" candidate_term: 9 candidate_status { last_received { term: 7 index: 9 } } ignore_live_leader: false dest_uuid: "a53441872c0c416b8e2f26f52399f536" is_pre_election: true
I20260809 15:53:13.270330 10926 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5dc95cd5fbb945e7af0f89cf2ec436c9" candidate_uuid: "c7aa762318cb4e97875333f10679a9f9" candidate_term: 7 candidate_status { last_received { term: 6 index: 8 } } ignore_live_leader: false dest_uuid: "20730d75ad954a82b126148f8152a9f3"